365 Data Science is an online educational career website that offers the incredible opportunity to find your way into the data science world no matter your previous knowledge and experience.
Originally from KDnuggets https://ift.tt/2IPeGlR
source https://365datascience.weebly.com/the-best-data-science-blog-2020/ai-and-automation-meets-bi
Welcome back to our blog post series where you’ll learn how we run machine learning projects. In the first part of this series you’ve already learned about the different phases of the project lifecycle. Today, we take a closer look at the first two very important phases: data collection and data preparation.
You should definitely read on if you plan to implement AI-supported tasks in your company, regardless of whether you are a project manager, engineer, or decision-maker.
1. How to automatically deskew (straighten) a text image using OpenCV
3. 5 Best Artificial Intelligence Online Courses for Beginners in 2020
4. A Non Mathematical guide to the mathematics behind Machine Learning
The first step in any machine learning project is to collect enough samples with the correct metadata, as this is the basis for our model for learning and prediction. But what data do we actually need? To find that out, we need to define the requirements first together with our customer. Let’s look at an example from our practice.
An umbrella federation for dance uses a Digital Asset Management system (DAM) as a central hub for images from all its members. These members are regional foundations and associations, professional dance schools, archives, stages, theatres, dance companies, and also single dancers — a diverse community which regularly uploads images from events, projects, and daily work to the platform. The images are used for the federation’s website, marketing, and social media campaigns.
What the dance federation needs is an automatic curation of the hundreds of images uploaded to the DAM every day. Are these images suitable for presenting the work of their members in a visually appealing way? The goal is to find aesthetic images faster when selecting new marketing collateral, ideally by simply using a filter option for “aesthetic” images. Our task is to classify the images as “aesthetic” or “unaesthetic”.
We cannot emphasize enough that it’s crucial to define the requirements in detail before starting to collect data, because this is the foundation for the entire project.
Now that we know the requirements, it’s time to gather the data for the training. We need two different categories of data:
Question is: Where do we get the data from and how much data do we need? Of course, we try to get the needed data directly from our customer first. Let’s say 10k images with labels are required. This should be sufficient to train a pretrained model via transfer learning.
In our case, the dance federation has 15,000 images whereof 1,500 have been already used as marketing collateral. We can regard these 1,500 images as aesthetic, and we label them accordingly. The other 13,500 images are unclear: they have not been used in any marketing activities, we don’t know if are made up.they can be considered as aesthetic or not. We take a subset of 5,000 images to add them to our dataset. To do so, we have to label them first.
These 5,000 images are labeled manually using workforce from Amazon Mechanical Turk. Since aesthetics is very subjective, it makes sense to have the images evaluated several times by different people and label the image according to the majority. After labeling is finished, we have 1,300 further images labeled with aesthetic and 3,700 images labeled with unaesthetic.
To make the data more balanced we need to get some more aesthetic data. We can search in dataset search engines like Academic Torrens or Google’s Dataset Search. We must bear in mind that available datasets can only be used if the license of the material (images as well as their metadata) allows commercial use. Many datasets are only allowed to be used for academic and research purposes.
But there are also open-access datasets, check out this compilation of dataset finders and free datasets. In our case we’ve found a good dataset of 900 images. These are aesthetic dance photos labeled with the keywords aesthetic and dance. We add them to our training data.
We have to admit that we did not reach the targeted 10k images, but we have 7,4k images which is a good amount to start with.
Out resulting dataset is very well balanced, we have the same amount of aesthetic and unaesthetic images. But in most projects a fully balanced dataset is not in reach. In those cases we accept small imbalances in the data rather than throwing away valuable data. The imbalance should not exceed a certain degree.
After we have collected all the data we need, we have to prepare it for training, which means cleaning the data and preprocessing it.
The collection process is quite complex and time-consuming, and the same applies to data preparation. Our data scientists spend most of their time and effort in a project on gathering the data, bringing it into the right shape, and refining it.
The complexity of the cleaning process highly depends on the amount of data sources we use, the amount of categories that need to be classified, and also on how complex those categories are.
In our example project we can mostly trust the data: the data is already very well balanced and we have trustable data sources. Therefore, only little cleaning steps are necessary. At first, we filter out and remove corrupt files (images which do not load). Secondly, we take a random look at the 900 images from the free dataset to check if they are labeled correctly. If we detect falsely labeled images, we remove them from the dataset.
To normalize the data we scale images to the appropriate size and convert the images to the right format.
Additionally, we augment the dataset by generating modified versions of each image. The goal is to introduce common variations the model should be aware of. Since our model should predict aesthetics we may not make any changes that could alter the aesthetics of the image. For example we do not crop images to retain the image layout.
Data augmentation should help the model not to get involved in false evidence. Let’s say we have many aesthetic images of a dancer in a red dress. This could be interpreted by the model in such a way that all images with red clothes are aesthetic. The solution would be to bring in some altered images by desaturating, changing colours, or adding black and white variants.
Now, we have collected all our data and we have prepared it for model training. Stay tuned for our next article of this blog post series where we dive deep into model evaluation and model training. Meanwhile, if you need someone helping you out with your AI project, just get in touch.
Data Collection and Preparation: Join us through the ML Project Life Cycle was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.
Innovative financial business models in fintech provide progressive products and services which make our everyday lives easier, by transacting our actions on the go.
Siri wasn’t really responsive, when I whispered to her, asking her to pay for my shopping at Earl’s Court Sainsbury’s. She was even quite indifferent, saying: “Sorry, that feature isn’t available, due to your device settings or region.” So I just used my fingerprint to activate the wallet app to make the payment.
Well, I still haven’t tried Monzo’s capabilities, using prepaid debit-cards and I understand that Monzo, just like Revolt, provides many more financial transaction features. However, I just thought it would be nice to connect Siri to the Wallet App and ask her to pay every time I needed to swipe my card, e.g. when buying coffee or travelling on the tube. The idea of linking Siri with your Wallet App sounds fine to me.
1. How to automatically deskew (straighten) a text image using OpenCV
3. 5 Best Artificial Intelligence Online Courses for Beginners in 2020
4. A Non Mathematical guide to the mathematics behind Machine Learning
Yes, there are lots of security issues to be addressed, but honestly, I just love the idea of ordering goods from Amazon, whilst talking to Alexa.
Anyway, I started visualising how nice it would be to have a special AI financial bot instead of a Wallet App. The finbot would be in charge of your bank account; would track your expenditure and alarm you if you decide to go beyond your spending limits. And not only this, your financial bot would help you make your payments and send copies of receipts into your mailbox. And it would be fun, if a finbot could support small talk about today’s market and every morning bring you highlights from the Business Press.
Well, this is from my perspective of an ordinary user. However, all this eventually might turn into something more valuable. An AI finbot could digest financial data from the stock markets and calculate revenue growth, profits, dividends, capital growth, share prices etc. and process this data to make recommendations and help with decisions about buying and selling bonds, shares and other financial products.
Having this in mind, I doubled my research and discovered some interesting tech innovations.
If brokers need software to do B2B and B2C sales of insurance policies, they can address PremFina www.premfina.com which is based in London. Another company called Morningstar www.morningstar.com provides a range of products and services that allow investors to tap into professional-grade financial research, which helps them to make better investment decisions. PitchBook www.pitchbook.com based in Seattle and Washington provides information about public and private equity markets and the software encourages investors to maximise on new business opportunities, using due diligence research on market intelligence, fundraising information, and thereby source new investments.
Then I thought it would be useful to look at investment in the fintech SME market, as it would be logical to see if there is anything happening in financing small fintech enterprises. Charlotte Crosswell, CEO of Innovate Finance, says: “Fintech lenders have certainly played their part in disbursing the lifelines of CBILS (Coronavirus Business Interruption Loan Scheme) to small businesses.” They have been very helpful in supporting SMEs with the capital, taking into account that UK entrepreneurs in the past hadn’t had enough access to financial resources because of a lack of credit. Charlotte says that was another challenge for those innovators who had come up with new digital businesses to address the needs of SMEs. “Challenger banks and alternative lenders such as Starling, iwoca, Funding Circle, OakNorth, Tide, Capital on Tap and Atom Bank, have all started since 2010 intending to disrupt the SME lending market,” — said Charlotte Crosswell.
Fintech lenders are already cooperating with 500,000 SMEs in the UK and it’s obvious what massive support they are giving to them for their growth.
Having read about this, I would like to point out the Bank of England’s “ New economy, new finance, new Bank” report. One of its priorities is to “help create an open platform to boost access to finance for small businesses and choice for households” (https://www.bankofengland.co.uk/-/media/boe/files/report/2019/response-to-the-future-of-finance-report.pdf?la=en&hash=34D2FA7879CBF3A1296A0BE8DCFA5976E6E26CF0)
The Bank of England says it will help to close the £22 billion SME funding gap in the UK, to support SMEs and “harness the power of their data by developing the concept of a portable credit file, to give greater access to more diverse and competitive financing options, including for global trade.”
Now, I was satisfied that investment in small fintech firms is doing quite well, but I was still curious if there are firms that are already doing anything to help users deal with their financial transactions and investments?
Drifting in the tomes of information and thinking about the security risks of linking your electronic wallet with the AI bot, I somehow bumped into the firm Onfido www.onfido.com which helps businesses, including those who provide financial services, to verify people’s identities. Machine learning provides face and character recognition, verifies passports, and checks ID cards, to help companies with fraud prevention.
Discouraged by these risk perspectives, that there is always someone who will want to defraud you, I was about to give up with the idea of a digital assistant to manage personal accounts.
What if the machine, which learned to deal with your finances, at some point decides to defraud you? What if an AI bot (who you thought was your bosom mate) teams up with other dodgy bots to swindle you or other human beings?
A firm called Plum Fintech registered at Companies House as Plum Fintech Limited says it uses Artificial Intelligence and behavioural science to support users’ interactions with their finances. The software analyses bank transactions and highlights savings they could invest into something else. Also, it advises about energy suppliers and utilities, making comparisons and proposing better deals. It is said that on average a customer saves £230 pa. However I experienced complete pleasure, when I came across a firm called Poly AI www.polyai.com This group of AI experts, who had worked for Facebook AI, Apple Siri and Google, after first meeting at the Machine Intelligence Lab at the University of Cambridge, decided to start Poly AI. They united around the delightful idea to revolutionise human-machine communication. They are teaching machines to converse with human-beings and to hold human-level conversations! I finally decided I found heaven!
These innovative fintech value-propositions are becoming a paradigm of the nearest future and as Michael Kent, the CEO of Azimo online remittance service, said that London is “starting to look like Silicon Valley” when it comes to tech investment. This inspires me to think that the bright day is coming, when we will be making our journeys on the tube paid by a personal AI finbot who will be sending us updates on our tablets and smartphones, including stock markets and digested financial news.
Anyway, the major challenge which needs tackling here, is the security of personal accounts. Is our personal data and access to financial accounts safe enough and properly encrypted, so it won’t be leaked or hacked? Can we rely on AI financial bots in terms of investments and can we trust the calculated potential gains? How accurate or reliable are the digital algorithms in the world of finbot machines, so that they effectively manage our funds? Many questions arise and many aspects of finbots need to be addressed in the future.
Admir Kurman
What if the machine decides to defraud you? was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.
Originally from KDnuggets https://ift.tt/35IqX4s
Originally from KDnuggets https://ift.tt/36Pv8L5
Hi! I am Susan Walsh – the Classification Guru – and I’m happy to join 365 Data Science in their great initiative: 365 Data Use Cases. I specialize in spend data classification. It also happens to be my favorite data use case and I am excited to share more about it with you in this article.
You can check out our video on the topic below or just scroll down to keep on reading.
Spend data classification is usually performed by the procurement department or the finance department in an organization and it takes a list of everything that the organization has bought from a given supplier over a certain period of time. Spend data usually contains the supplier name and normally an invoice description of the goods that they bought.
Let me give you an example.
Let’s say Staples is the supplier name and the goods your company has purchased are pens, pencils, paper, paper clips, etc. Or if you’re in manufacturing, employees could have bought nuts, bolts, and screws, so that’s what’s listed within the company’s spend data.
The normalization of data is crucial for data classification within the procurement process. And, as a first step, you need to standardize data. So, what I do is take the file with all the suppliers and standardize them to one term. The reason behind this is that most of the time procurement management executives don’t actually know how much they’re spending with one supplier as the latter might have different divisions or different countries and all that data is piled up together. In other words, it’s all in different formats.
A really great example is IBM. There are plenty of different variations – IBM, IBM Inc, IBM Ltd., I.B.M, etc. So, they should all be standardized to just IBM.
As the next step, I go through the list and I will classify the data using a taxonomy. Broadly speaking, the result is a table with one to three levels but there could possibly be a fourth and a fifth level as well. Generally, it will be the set of classifications that I will use in the data set. To illustrate with the Staples example – facilities might be level one, office supplies might be level two and paper might be level three.
Then I go through the categories and perform data standardization and data classification once again. This is necessary because there are numerous ways to describe the same product on invoices and descriptions. As a matter of fact, screws are a great example of that. They could be listed as screw, screws, SCR, SCW, SCWs… As you can see, there are loads of different versions.
Data Classification enables me to show the company’s procurement analyst exactly how much they’re spending on each product.
And what’s the benefit of that?
Well, first of all, it translates into cost savings. So, maybe the procurement manager doesn’t realize how many screws or paper they were buying. Spend data classification will help them discover that they could be negotiating better rates with their suppliers. Or maybe they will arrive at the following conclusion: “Oh, Supplier A is charging us this, but Supplier B’s charging us that. So, we need to start using supplier A more.” Another benefit is that spend data classification can also show you how many suppliers per category you have. This is very important, especially if it turns out your company has 20 vendors for office supplies and you don’t need that many. Certainly, you could negotiate a better rate with two or three suppliers that you have.
So that was our whistle-stop tour of spend data classification. I hope you enjoyed it and it helped you learn the basic idea behind spend data classification.
If you have any questions, feel free to contact me on LinkedIn. I also have a YouTube channel – The Classification Guru, where you can find more resources on classification, taxonomy, normalization.
And if you’re new to data science and want to fully understand and distinguish between the various terms and processes in the field, check out the 365 Intro to Data and Data Science course.
The post 365 Data Use Cases: Data Science and Spend Data Classification with Susan appeared first on 365 Data Science.
from 365 Data Science https://ift.tt/2UN0HPT
Originally from KDnuggets https://ift.tt/2KlceUS
Originally from KDnuggets https://ift.tt/3kKjndG
Originally from KDnuggets https://ift.tt/32Xmlpk
Mobile apps have brought innovations and modified the way of working, shopping, and communicating. During 2020 and 2021, several different technological progressions influenced app development; it finally helped small, moderate, and large enterprises in expanding business growth.
Worldwide by 2021, there will be 3.8 billion smartphone users, and it has been predicted that the count of the same will gradually increase year by year.
Source: Statista
Using the mobile app integrated with advanced technologies can help enterprises to achieve their goals quickly.
1. How to automatically deskew (straighten) a text image using OpenCV
3. 5 Best Artificial Intelligence Online Courses for Beginners in 2020
4. A Non Mathematical guide to the mathematics behind Machine Learning
The mobile app development sector impacted the approach of advanced technologies. These trending and leading technologies have made significant changes in app development, and this finally offers beneficial outcomes to startups, SMEs, and large enterprises.
Now the Small businesses are also constantly realizing the significance of incorporating the technology-based app into business processes. That’s why now most of the enterprises prefer availing mobile app development services from the reliable and the best Mobile App Development Company India.
Let’s discuss how technologies in mobile app development will make storms in 2021.
Source: Google
Artificial Intelligence (AI) and Machine Learning (ML) wholly remodeled the face of mobile app development, and it also has been predicted that these top mobile app technologies will accelerate the growth of businesses up to much extent.
The amalgamation of AI with mobile apps saves businesses money and also enhances user engagement. Nowadays, most businesses prefer to integrate AI-powered chatbots in the apps, and the usage of such an app is increasing day by day.
37% of enterprises employ Artificial Intelligence (AI) at the workplace.
Advantages of using AI in Mobile App Artificial Intelligence helps you complete monotonous work promptly.
Source: Google
The increasing number of mobile devices forms diversified opportunities for IoT technology. IoT is a network of Internet-connected devices that gives automated control and ease to end-users.
The adoption of IoT in smartphone mobile apps benefits remotely control smart gadgets. Moreover, mobile apps powered by IoT allow you to connect bands, smartwatches, and other wearables to Smartphones.
The worldwide spending on the Internet of Things (IoT) is projected to reach $1.1 trillion in 2022.
Source: Google
An AR app is a software application that unites digital visual, audio or other content into the real-world environment. The mobile app using VR and AR technologies are creating unforgettable experiences for mobile users and helping companies in increasing sales.
In many industries such as healthcare, public safety, gas and oil, tourism and marketing, AR & VR, mobile app development trending technologies are used for training work and customer applications. Worldwide the market of Mobile App Development for Augmented Reality and Virtual Reality Apps is rising.
In the United States (US), 77% of users use VR/AR-based Smartphones.
Source: Statista
Source: Google
Most of businesses prefer using chatbots in the business-mobile apps; this mainly helps in expanding customer experience. By using chatbots in the mobile app, you will be able to grant 24*7 services to your clients.
Seeing the usage of chatbot in mobile app development, I can say that the utilization of this will get double in the upcoming years.
85% of all client interactions will be managed without a human agent by 2020. The chatbot market size is predicted to reach around $1.25 billion in 2025.
Source: Google
Blockchain is one of the famous mobile app trends; it transformed mobile app development and benefited enterprises. This technology can be integrated into mobile apps for intensifying security, tracking, and quality controls.
Various payment apps prefer using Blockchain technology for warranting secure and quicker transactions. Various industries like healthcare, tourism, retail, finance employ this technology in mobile apps. It has also been predicted that in upcoming years the usage of such a technological trend will increase from a high pace.
By 2025 the global blockchain technology revenue (in size) expected to reach over $39 billion.
Advantages of using Blockchain in mobile apps
Famous Mobile Apps Using Blockchain
Source: Google
5G technology is the latest monotony of the mobile world which will boost the agility and responsiveness of the wireless interfaces. With the help of this mobile app development trending technology, the total amount of data that is transmitted over the wireless broadband bonds can travel at tremendous rates like 20 GBps with high network speed.
It is not just one of the top technology trends in mobile app development but also highly significant in the upcoming 2021. Well if you are thinking to include these top-level technologies in your mobile app then do hire mobile app developers as this will help you to get the better result.
By 2024, projections predict that there will be around 1.9 billion 5G subscriptions worldwide.
Advantages of using 5G technology in mobile apps
Source: Google
Cloud technology is the trending mobile app development technologies that help enterprises to store a huge amount of data. This technology, when united with mobile apps, prolongs the storage capacities and boosts productivity and collaboration.
Well, several mobile apps with more giant databases use the various cloud technologies such as Amazon Web Service (AWS), Microsoft Azure as their cloud computing backup.
The storage, hosting, and computing cloud services market was predicted to be $126 billion in 2017 and was forecasted to reach $163 billion in 2021.
Advantages of using Cloud technology in Mobile App
Source: Google
M-commerce is used in wireless handheld gadgets like mobile phones and tablets to manage business transactions online, including buying and selling of products, online banking.
The use of m-commerce activity is growing day by day; that’s why now most prefer employing this technology in the mobile app.
By 2021, the share of mCommerce in all e-commerce is foreseen to grow to 72.9%.
The enterprises use mobile apps for doing multiple tasks like selling, buying, promoting service. But if I talk about business growth in 2021, then startups, SMEs, and big enterprises should focus on building and using the technology-based mobile app. This will not only help organizations in making the correct decision but will also help businesses in boosting productivity.
If you also want to develop the leading mobile app, then I will suggest you hire mobile app developers from the top Mobile app development company India. This will help enterprises in getting innovative mobile app development solutions.
Mobile Tech Trends 2021: Era of Immersive Technologies was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.
365 Data Science