Top Stories Nov 30 Dec 6: Why the Future of ETL Is Not ELT But EL(T)

Also: AI, Analytics, Machine Learning, Data Science, Deep Learning Research Main Developments in 2020 and Key Trends for 2021; Introduction to Data Engineering; Data Science History and Overview; Introduction to Data Engineering; Object-Oriented Programming Explained Simply for Data Scientists

Originally from KDnuggets https://ift.tt/3oxO4VX

source https://365datascience.weebly.com/the-best-data-science-blog-2020/top-stories-nov-30-dec-6-why-the-future-of-etl-is-not-elt-but-elt

Dark Data: Why What You Dont Know Matters

In his latest book, a leading statistician Dr. David Hand explores how we can be blind to missing or unseen data and how, in our rush to be a data-driven society, we might be missing things that matter, leading to dangerous decisions that can sometimes have disastrous consequences. Download this free chapter now.

Originally from KDnuggets https://ift.tt/3qBB2sf

source https://365datascience.weebly.com/the-best-data-science-blog-2020/dark-data-why-what-you-dont-know-matters

10 Real World Problems Effectively Solved by the AI

Smart assistants building impressive gadget at our command and terminator machines destroying the world, these are the first pictures that come into our mind when someone mentions the name AI.

In real life, however, AI is more positive and grounded. While AI has not reached the level of assembling gadgets yet, it has made the world a better place by solving many complex problems.

In this blog, we will talk about ten such real-world problems effectively solved by AI. Let’s begin:

Machine Learning Jobs

1. Online Shopping Made Easy

Shopping online for a product without knowing its name was a nightmare a few years back. It took hours to find that product from the catalog.

However, all thanks to AI trends like predictive technology, things are entirely different these days. When you search for an item even with a vague query, thousands of search results appear in the blink of an eye.

We even get suggestions based on our searches like these search engines can reading our mind. It appears that everything we want is not too far anymore.

Companies like CamFind are even taking this one step further by letting you identify objects by just clicking their picture. We are not far from the day when consumers will be able to shop items just by clicking their picture.

2. Consumer Queries Resolved Faster & More Accurately

The biggest challenge among businesses is to ensure the queries of their consumers are resolved on time. Things become even more tough when you are a big organization with millions of customers.

India’s leading finance bank, HDFC was facing the same difficulty. So, they developed EVA (Electronic Virtual Assistant), an AI-based chatbot which can provide simple answers in less than 0.4 seconds by collected knowledge from thousands of resources.

The virtual assistant has been responsible for addressing 3 million queries, interacting with half a million unique users, and holding over half a million conversations.

So, you can see how much impact can adopting AI in customer support can create.

Trending AI Articles:

1. How to automatically deskew (straighten) a text image using OpenCV

2. Explanation of YOLO V4 a one stage detector

3. 5 Best Artificial Intelligence Online Courses for Beginners in 2020

4. A Non Mathematical guide to the mathematics behind Machine Learning

3. Frauds Prevented

Tracing and preventing frauds have always been a big challenge that AI is helping us overcome. Organizations are using AI solutions to prevent fraud and enhance security in a number of sectors.

Companies like MasterCard and RBSWordPay are using AI and deep learning to recognize doubtful transaction patterns and thwart card frauds. Other industries are using the same approach to prevent such cons from happening. It has helped them save millions of people from getting conned off their hard-earned money.

4. Farmers Producing More Crops with Less Resources

According to stats, we will need to produce 50% more food by the year 2050 because we are consuming a lot. For this, farmers will have to produce crops while using their resources wisely.

Yet the question is how!

Blue River Technology, the subsidiary of American corporation John Deere figured out the solution by creating a robot called See and Spray. The robot uses object detection (a computer vision technology) to monitor and spray weedicide on cotton plants in precise amount.

Not only this, an agriculture startup called PEAT has created an app called Plantix that makes use of image recognition technology to identify potential defects and provide techniques, tips, and solutions.

So, you can realize how these little things can improve the agriculture industry and help us meet the crop requirements.

5. We No Longer Have to Worry about Diseases

What if we can identify and prevent diseases even before they occur? We will live a lot longer.

You might have read this a hundred times till now. Yet the only question you find yourself asking is: “how is that possible?”

The answer is: With the power of AI.

A healthcare organization called Cambio Health Care has developed a clinical decision support system that warns a physician in advance when a patient is at the risk of having a heart stroke so that they can prevent it from happening by taking appropriate measures.

Another company called Coala life also created a digitalized device that can detect Cardiac diseases. Even another company called Aifloo is creating a system that will keep track on patients’ health in nursing homes and healthcare centres.

These examples prove that we are not far from days when we no longer have to worry about finding a cure for new diseases. Instead, we can use existing cures to save more lives.

6. E-learning is Much Interactive & Fun Now

With an expected worth of over $325 billion by the year 2025, e-learning is one of the fastest-growing industries. However, keeping up with its pace is a real challenge.

There are millions of e-learning courses available. How many you think do receive the attention they deserve?

Thankfully, with the help of AI, designing interactive courses is easy these days. With features like learning pathways, personalized tutoring sessions, content analytics, targeted marketing, automatic grading, and real-time questioning — AI is changing the face of e-learning.

No wonder why platforms like Duolingo (30 million registered users) and Massive Open Online Courses (101 million registered users) are getting so popular.

Another interesting example of AI playing a critical role in elearning is an online PM training website I recently found on the internet. It uses AI and behavior science for designing interactive courses of online training.

So, AI is truly reforming the fate of elearning and online education, making it more fun and an interactive experience.

7. Solving Puzzles is No Longer a Challenge

We all love solving puzzles. However, staying stuck for hours while trying to solve them is something we would never want.

For example, I am addicted to Scrabble. However, it used to take a toll me because I could not think of enough words. The most frustrating part was that I used to lose every time I played.

However, with the scrabble word finder I recently found online, losing while playing this game is out of the question because I can easily find the best scoring words.

Not only this, AI is even helping historians solve ancient puzzles to restore and recreate historical artifacts from photos of fragments. Things like this can be a great contribution to mankind.

So, we are no far from days when there will no mystery that AI cannot uncover and no puzzle it can’t solve.

8. We are Learning through Games

Can games teach us something?

For most people, the answer to this question will be “No”.

However, all thanks to AI, this notion is going to change soon. Games are using AI to train players and improve their skills.

The game F.E.A.R (First Encounter Assault Recon) uses AI to train players. The actions of the opponent AI in this game are so unpredictable that it will train you throughout the game to never make the same mistakes. You get better as the game gets harder. No wonder people like it.

Another interesting example is Google DeepMind’s AlphaZero which is taking the game of chess to the next level by defeating grandmasters like Gary Kasparov and Vladimir Kramnik. Similar is happening in the case of AlphaGo.

So, all thanks to AI, games are teaching us to expand our limits and improve like we never did before.

9. Fighting Hate Speech and Trolls on Social Media has Become Easy

Fighting hate speech and trolls on social media has been one of the biggest challenges to date. Not only tracking these trolls and hatemongers is difficult, but there is also little you can do about them.

Twitter was facing the same issue. So, it built an AI that can easily identify hate speech and terrorist language with the help of deep learning, machine learning, and natural language processing.

As a result, the social media mogul has banned over 300,000 terrorist accounts. Even Facebook and Instagram have also started using AI to over spammers, hatemongers, and trolls.

So, we are not far from days when fighting hate speech and trolls will not be a challenge anymore.

10. Athletes are Able to Enhance Their Performance

Being a sportsman demands sheer dedication and endless practice. However, sometimes just practicing alone is not enough. You have to combine it with the right strategy.

AI is helping athletes push their limits and accomplishing the impossible by breaking the game into small chunks and then studying it closely with the help of machine learning algorithms.

Not only this, the combination of sensor technology and AI is helping coaches improve players’ techniques. It is also helping them prevent player injuries by keeping a close check on the levels of strain and exertion players are experiencing.

All these little things are helping players achieve impossible milestones and take their performance to whole new level.

In a Nutshell

While the concept of AI might still seem fiction, it is already changing the world. These ten real-world examples are proof. It is helping us achieve what earlier seemed impossible.

The trend will continue to revolutionize our lives in the future. So, if you are a beginner who wants to start a career in AI or a business looking forward to incorporating the trend into your operations, this is the right time.

Are you wondering where to get started? These 5 AI courses for beginners may be the right first step.

Don’t forget to give us your ? !


10 Real World Problems Effectively Solved by the AI was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

Via https://becominghuman.ai/10-real-world-problems-effectively-solved-by-the-ai-485f2f0c555c?source=rss—-5e5bef33608a—4

source https://365datascience.weebly.com/the-best-data-science-blog-2020/10-real-world-problems-effectively-solved-by-the-ai

Preparing for the Great Reset and The Future of Work in the New Normal

The world is changing incredibly fast today. I have talked frequently in previous articles about the rate of change we are going through…

Via https://becominghuman.ai/preparing-for-the-great-reset-and-the-future-of-work-in-the-new-normal-a3c195b7eccf?source=rss—-5e5bef33608a—4

source https://365datascience.weebly.com/the-best-data-science-blog-2020/preparing-for-the-great-reset-and-the-future-of-work-in-the-new-normal

Essential Math for Data Science: Probability Density and Probability Mass Functions

comments By Hadrien Jean, Machine Learning Scientist   In the chapter 02 of Essential Math for Data Science, you can learn about basic descriptive statistics and Read more »

Originally from KDnuggets https://ift.tt/2IoNKth

source https://365datascience.weebly.com/the-best-data-science-blog-2020/essential-math-for-data-science-probability-density-and-probability-mass-functions

The Ultimate Guide to Data Engineer Interviews

If you are preparing for data engineering interviews, then follow these technical recommendations regarding your resume, programming skills, SQL acumen, and system design problem-solving, as well as the non-technical aspects of your upcoming interview session.

Originally from KDnuggets https://ift.tt/2Lh7AYr

source https://365datascience.weebly.com/the-best-data-science-blog-2020/the-ultimate-guide-to-data-engineer-interviews

Understanding Confusion Matrix

When we get the data, after data cleaning, pre-processing and wrangling, the first step we do is to feed it to an outstanding model and of course, get output in probabilities. But hold on! How in the hell can we measure the effectiveness of our model. Better the effectiveness, better the performance and that’s exactly what we want. And it is where the Confusion matrix comes into the limelight. Confusion Matrix is a performance measurement for machine learning classification.

This blog aims to answer following questions:

  1. What the confusion matrix is and why you need it?
  2. How to calculate Confusion Matrix for a 2-class classification problem?

Today, let’s understand the confusion matrix once and for all.

What is Confusion Matrix and why you need it?

Well, it is a performance measurement for machine learning classification problem where output can be two or more classes. It is a table with 4 different combinations of predicted and actual values.

Artificial Intelligence Jobs

It is extremely useful for measuring Recall, Precision, Specificity, Accuracy and most importantly AUC-ROC Curve.

Let’s understand TP, FP, FN, TN in terms of pregnancy analogy.

True Positive:

Interpretation: You predicted positive and it’s true.

You predicted that a woman is pregnant and she actually is.

True Negative:

Interpretation: You predicted negative and it’s true.

You predicted that a man is not pregnant and he actually is not.

False Positive: (Type 1 Error)

Interpretation: You predicted positive and it’s false.

You predicted that a man is pregnant but he actually is not.

Trending AI Articles:

1. How to automatically deskew (straighten) a text image using OpenCV

2. Explanation of YOLO V4 a one stage detector

3. 5 Best Artificial Intelligence Online Courses for Beginners in 2020

4. A Non Mathematical guide to the mathematics behind Machine Learning

False Negative: (Type 2 Error)

Interpretation: You predicted negative and it’s false.

You predicted that a woman is not pregnant but she actually is.

Just Remember, We describe predicted values as Positive and Negative and actual values as True and False.

How to Calculate Confusion Matrix for a 2-class classification problem?

Let’s understand confusion matrix through math.

Recall

Out of all the positive classes, how much we predicted correctly. It should be high as possible.

Precision

Out of all the positive classes we have predicted correctly, how many are actually positive.

Accuracy

Out of all the classes, how much we predicted correctly, which will be, in this case 4/7. It should be high as possible.

F-measure

It is difficult to compare two models with low precision and high recall or vice versa. So to make them comparable, we use F-Score. F-score helps to measure Recall and Precision at the same time. It uses Harmonic Mean in place of Arithmetic Mean by punishing the extreme values more.

I hope I’ve given you some basic understanding on what exactly is confusing matrix. If you like this post, a tad of extra motivation will be helpful by giving this post some claps ?. I am always open for your questions and suggestions. You can share this on Facebook, Twitter, Linkedin, so someone in need might stumble upon this.

You can reach me at:

LinkedIn : https://www.linkedin.com/in/narkhedesarang/

Github : https://github.com/meetvansjaliya

Thanks for Reading!

Don’t forget to give us your ? !


Understanding Confusion Matrix was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

Via https://becominghuman.ai/understanding-confusion-matrix-eb6f0f662c3a?source=rss—-5e5bef33608a—4

source https://365datascience.weebly.com/the-best-data-science-blog-2020/understanding-confusion-matrix

Augmented Intelligence for Intelligent Entrepreneurs TL;DR

or Augmented Intelligence vs Artificial Intelligence round 1.

It’s Monday afternoon in the office. An email pops up in my Gmail from an international student organization. They’re asking me to give a keynote lecture about “innovation in technology.” Referred by the Israeli Embassy? Well, there’s nothing I enjoy more than representing the motherland — I press reply all:

“I am in.”

In my last couple of years as the CEO of Spartans and Jenna.ai, I’ve done dozens of these tech keynotes. But their broad topic — innovation in technology — makes things a bit more difficult. Is it worth it to do the normal AI spiel?

Buzzwords and trends come and go in the tech world. Years ago it was Big Data, and then Chatbots & Blockchain, and now it’s AI all around. Quantum Computing is getting warmer.

That’s great, but…

What would a hungry-to-learn crowd of future tech entrepreneurs find actually interesting? That’s the question I asked myself when I started trying to build presentations. I abandoned a few before I settled, finally, on one topic.

Augmented Intelligence.

Artificial Intelligence Jobs

The Basics

“Augmented Intelligence,” “Intelligence Amplification,” or “Enhanced Intelligence.” Even if you haven’t heard these terms before, you’ve probably used the tech. They all mean the same thing: the empowerment of human intelligence using technology. It’s a broad field, and it’s growing fast.

According to Wikipedia, early computer pioneers first proposed the idea in the 1950s and 1960s. That’s quite a long time ago — so why this renaissance all the sudden?

Simple answer: Artificial Intelligence failed to deliver.

Now, hold your horses and put down your burning torches ??. AI is amazing, and it can be a great tool. Without it, some companies wouldn’t exist at all. Here’s a personal anecdote. In the last almost 4 years, we at Spartans have worked with AI on a daily basis. Trust me, I know all about it.

But we can’t forget that AI has its limitations. Sometimes, it just can’t replace brick-and-mortar human operation. AI gets a lot of hype, and it might not meet your expectations. There’s nothing worse than the mismatch between the need and tech.

So, what has Augmented Intelligence got to do with it?

Trending AI Articles:

1. How to automatically deskew (straighten) a text image using OpenCV

2. Explanation of YOLO V4 a one stage detector

3. 5 Best Artificial Intelligence Online Courses for Beginners in 2020

4. A Non Mathematical guide to the mathematics behind Machine Learning

Augmented Intelligence vs Artificial Intelligence

Some positions will disappear and be totally replaced by AI. Others will be empowered.

What is AI, anyway? Let’s start with the obvious. When we refer to Artificial Intelligence, we refer to a ton of capabilities. In a nutshell, AI’s intention is to replace a certain function human are currently doing. Fields like Machine Learning, Computer Vision, Deep Learning, and others all use AI.

What is Augmented Intelligence, and how is it different? Augmented Intelligence references an approach, rather than a specific mode of tech. It enriches human capabilities and supports the process of decision making or action. Augmented Intelligence isn’t here to replace the human factor, but enhance it.

But, let’s be a bit more precise, shall we?

Human and Machine by Definition

Augmented Intelligence is the design pattern for a human-centred partnership. People and AI work together to enhance human cognitive performance. This means AI can help us with learning, decision making, and new experiences.

So yes, Augmented Intelligence is a reference to AI, but specifically human-centric AI. Augmentation is growing to be the most popular type of AI implementation. Don’t believe me? You don’t have to. Just check out this chart.

According to Gartner, Augmentation will be by far the fastest-growing type of AI. Will hold up to 44% of all AI implementations.

So yes, the modern reference to Augmented Intelligence is a reference to AI but human-centric AI. This type of AI that is in some cases referred simply as Augmentation will be the most popular type of AI implementation. Don’t believe me? you don’t have to — check the chart above.

So that’s it? well not exactly.

When referring to the product/implementation cycle, we see some difference in the flow of work to support the approach…

Pipeline in Augmented Intelligence

The Augmented Intelligence pipeline is built specifically to support humans. It’s designed to “enjoy” and learn from its feedback from humans.

We have 3 main actors in this process:

  1. Historical and new data
  2. The human factor
  3. Artificial Intelligence

The symbiosis begins with the AI’s initial training. We feed the AI structured and unstructured historical data from human operations.

The more data, the merrier.

Once the AI module becomes relatively accurate, we move to testing and deployment. Then we can take a look at the direct impact of the AI solution. How effective will it be for the team or person using it?

And here come those humans again.

The human team constantly interacts with both the data and the Augmented Intelligence. When the AI works correctly, the human team receives insights to help them work better and faster. Everyone contributes by inputting passive and active feedback into the system.

What is passive feedback? All humans need to do is use the system in their own way. When we make a decision to use, or not use, a certain recommendation, the AI knows what it needs to improve.

What is active feedback? In some systems, humans can give feedback by directly telling the AI how it’s doing. Does the AI’s score or answer make sense? How much? Even a thumbs up or thumbs down can give a lot of actionable input.

Augmented Intelligence in Real Life

Here are some common examples of Augmented Intelligence. You’ve probably used it before, even if you weren’t aware at the time — most people have.

Gmail — Fast Reply Recommendations

You might already be familiar with this example. Google launched “Smart Reply” in their Inbox app, and it became a huge success. The AI suggests email replies based on your past activity, and you can choose with just one click. This feature is fast, easy, and intuitive.

Gmail Smart Reply Augmented Intelligence Solution
  • The data — an unimaginable amount of historical emails
  • The AI — Trained Neural Network
  • Human — well, you

Jenna —Candidate Screening & Scoring

The second example, Jenna.ai, is one of our daughter ventures at Spartans. So yes, I am definitely biased, but that doesn’t mean it’s not a great system.

The platform interacts textually or vocally by using NLP/U with potential job candidates. It gathers the relevant information about each one and then predicts if they’re a good fit. Jenna also conducts fully autonomous decisions to filter irrelevant candidates.

Jenna provides the human staff with ML-based insights and data. It gives them the tools to make the decision or action that is best for the company. With a simple thumbs up and thumbs down, recruiters can give Jenna feedback on what to improve.

  • The data: Pre-gen berated data set + newly generated data loop
  • The AI: Series of ML modules and NLP/U engine
  • Human: The recruiter team and the relevant candidates

Gong — Candidate Screening & Scoring

Sales intelligence platform “Gong” recently raised $200 million (show.me.the money!). The platform offers AI-driven insights for sales teams around the world with key clients such as Linkedin, Pinterest and others.

The platform provides a series of critical insights to and about the sales process with a clear goal to increase win % and ROI.

No replacement of the human sales reps (at least for now) but an ongoing human-AI relationship which includes analysis of calls and textual communication to generate actionable insights for the sales cycle.

Main takeaways

You made it to the bottom line. As promised, you deserve the TL;DR version, so here it goes:

  • Augmented Intelligence is an approach and point of view, not a specific technology.
  • It doesn’t want to replace humans, but help them get better at certain tasks.
  • Most AI implementations today are Augmentations.
  • Some examples include Gmail’s reply suggestions and Jenna.ai’s employee screening process.
  • There are three components in the product chain → Data, Humans, AI
  • Humans and AI working well together → Big success. I like! (Read aloud like Borat)
Photo by Possessed Photography on Unsplash

Augmented Intelligence, in many ways, is the healthiest way to look at our relationship with AI. It’s collaborative tech, and it doesn’t have the same unrealistic goal of simply replacing humans. We can acknowledge the strengths and weaknesses of both AI and humans, and use that to our advantage. Disruptive innovation can start right here.

The base assumption for Artificial or Augmented Intelligence is just that: Intelligence. So use it wisely.

Not too bad of a keynote is it?

Don’t forget to give us your ? !


Augmented Intelligence for Intelligent Entrepreneurs TL;DR was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

Via https://becominghuman.ai/augmented-intelligence-for-intelligent-entrepreneurs-tl-dr-f7e6678e992e?source=rss—-5e5bef33608a—4

source https://365datascience.weebly.com/the-best-data-science-blog-2020/augmented-intelligence-for-intelligent-entrepreneurs-tldr

Design a site like this with WordPress.com
Get started