365 Data Science

A beginner’s guide to train and deploy machine learning pipelines in Python using PyCaret.

Originally from KDnuggets https://ift.tt/3gbz24N

Also: Everything you need to become a self-taught #MachineLearning Engineer ; SQL Cheat Sheet (2020) – a useful cheat sheet that documents some of the more commonly used elements of SQL;

Originally from KDnuggets https://ift.tt/2ZrOB2y

Caserta is offering a limited number of virtual pro-bono data and analytics workshops conducted by industry leaders Joe Caserta and Doug Laney exclusively for eligible senior leadership. Learn more and sign up now.

Originally from KDnuggets https://ift.tt/2Xj6H3R

AI and Machine Learning are two buzz words that everyone hears almost every day. But when we see the job requirements there is a miss alignment between what we actually see in the news and what companies require. Why is that? Let’s get it by learning what lies under the hood.

How a machine can have intelligence! Let’s get it through some history and a comparison with human anatomy.
In 1950 Alan Turing published an article named “Computing Machinery and Intelligence”. He started his article with this quote:

I propose to consider the question, can a machine think?

In 1970 when scientists thought of something like Speech Recognition or Image Recognition they had to do maths on paper with pencils and they made books of lookup tables. Which took an immense amount of manpower and time. Then they though if they could somehow make the machine learn all the data then it can make calculations on its own.

A computer scientist Geoffrey Hinton almost 40 years ago, was obsessed with figuring out how the human mind works. He studied physiology, the anatomy of our brain, psychology, and then got into more of a computer science approach to modeling the brain, called Neural Network in Artificial Intelligence.

Needless to say that Artificial Intelligence has been inspired by human anatomy. The reason why Geoffrey Hinton is called the godfather of AI in modern times because, Artificial intelligence is mostly about artificial neural networks and deep learning. But this is not how it always was. In fact, for most of its six-decade history, the field was dominated by symbolic artificial intelligence, also known as “classical AI,” “rule-based AI,” and “good old-fashioned AI.”

AI is a huge section to combine in a small diagram. But I tried to picture it this way:

Now, there are basically two approaches for a Machine to be Intelligent.

1. Symbolic Based:

Symbols play a big role in the human imagination.

Suppose I tell you that, I am writing this article lying on my sofa with my laptop on my lap. I am holding a pen on my right hand and swinging it to think about how to arrange this article well.

I am sure you have made an image of my situation on your mind and if you know me personally you may have pictured me as well. Being able to communicate with symbols is one of the main things that make human intelligent.
The early pioneers of AI believed that

“every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.”

Trending AI Articles:

1. AI for CFD: Intro (part 1)

2. Using Artificial Intelligence to detect COVID-19

3. Real vs Fake Tweet Detection using a BERT Transformer Model in few lines of code

4. Machine Learning System Design

Many of the concepts and tools you find in computer science are the results of these efforts. Symbolic AI programs are based on creating explicit structures and behavior rules.

An example of symbolic AI tools is object-oriented programming. OOP languages allow you to define classes, specify their properties, and organize them in hierarchies. You can create instances of these classes (called objects) and manipulate their properties. Class instances can also perform actions, also known as functions, methods, or procedures. Each method executes a series of rule-based instructions that might read and change the properties of the current and other objects.

In the early days of AI Computer Vision and robotics were tried to get implied using symbolic AI. As, you can see a flowchart of a robot car designed with symbolic AI. Let,s take an example of Computer Vision. Say you have a picture of your cat and want to create a program that can detect images that contain your cat. You create a rule-based program that takes new images as inputs, compares the pixels to the original cat image, and responds by saying whether your cat is in those images.

So, this was a symbolic approach to solve a computer vision problem. Now with the evolve of Neural Network, Computer Vision is being designed with Neural Network more previously Deep Learing.

One of the main differences between machine learning and traditional symbolic reasoning is where the learning happens. In machine- and deep-learning, the algorithm learns rules as it establishes correlations between inputs and outputs. In symbolic reasoning, the rules are created through human intervention. That is, to build a symbolic reasoning system, first humans must learn the rules by which two phenomena relate, and then hard-code those relationships into a static program.

Researchers realized that it’s really hard to handcraft all the rules and they need algorithms to extract information from data.

2. Data-Based (Machine Learning):
Machine learning is more of a data-driven side of Artificial Intelligence. It’s all about recognizing patterns from a huge amount of data.

Just like how we human practice more we get better at anything, i.e. Playing music, Singing, playing a game, coding etc. machine also gets better when it keeps dealing with various kind of data with different patterns.

Statistical Learning:
In this part, a machine performs on the statistics data or supervised learning. The more we introduce it with various languages, emotion etc. of a human speech, the more fluently it recognizes a new speech.
Now, Human can talk and recognize when someone talks, that is the part of Speech Recognition.
Human can read and write a text, that is part of NLP.
Let’s see a quick example diagram of NLP.

So, we make models with the help of some algorithms and feeding the algorithm data. The more robust our model become the more accurate our machine becomes on recognizing a speech.

Now, there is also Neural Network. Neural network is inspired by Human Brain.
Human Brain is made of zillions of neurons. These neurons capture data, process them, and store relevant information.

As, we can see from the comparison, a neuron in Neural network also gets data (x1, x2, …, xn) process them through a function and store an output just like a human brain does with its dendrites (inputs), cell body (f(n)) and axon (output).
A combination of several neurons makes a neural network. And when the neural networks are far more complicated and work on different levels it’s called deep learning.

Deep Learning has several sectors like CNN (Convolution Neural Network), RNN (Recurrent Neural Network).
These all depend on recognizing patterns from data. While human can recognize 2/3 dimensions from data for a machine it can be 9–10.

And when it’s come to calculation computers can beat humans. Because intelligence can’t beat calculations.
Example: World Chess grandmaster Garry Kimovich Kasparov least against an IBM program in a classic chess game under tournament rules.
Auto driving car by google uses laser and other sensors to make a 3d model of the road so that it doesn’t make an accident.

Now that we have learned a bit of history and brief knowledge about AI. Let’s have a quick look at why the news that we see about AI and what companies require in their job requirements are misaligned.

Let’s first see how these AI, ML has become a buzz word.

In 1996, engineers used to mine data from databases for knowledge discovery. Data Mining: The practice of examining large databases in order to generate new information.
In 2001 William S. Cleveland took it to another level. He thought that if we can make Computer to do the calculations of data we can make more in less time. So, Data Science = Computing + Data Mining/Statistics.
From, 2003 the websites has changed from static to more dynamic where we can upload data. In 2003 My Space, 2004 facebook, 2005 youtube. It enabled us to contribute to the internet and leave our footprint in digital landscape. That’s called internet.
That was lot of Data and too much to handle by regular technologies.That introduced Big Data. And also gave us important perspectives like getting insights and analytics from data. So, that’s when we started getting better results from the machine and started seeing the result of AI in our day to day life. and thus AI and ML news is everywhere.

But, we often see big companies posts job for Data Analyst, Data Scientist, Machine Learning Engineer, Deep Learning Engineer, etc. What do these actually mean? Before going to the hierarchy of need, let’s see a diagram that describes the steps of Data science.

Now these, steps are needed for a complete data analysis and getting meaningful results. So’ now let’s see the steps according to the hierarchy of need.

So as we can, the basic steps that we saw in the diagram for NLP the skeleton are mostly the same. We have to collect data, process the data, and get a better output. But, the intelligence lies here in between.
In big companies, they have a dedicated team for each step who tries to make it better, optimize it, and maximize its performance. In smaller companies, there can be less team and that team is responsible for 2–3 steps of the procedure. In startups, it may be only a person who does all of that on a small scale. So it actually varies what a role can be working on the Data Science field.

I hope this article helped to get an overview or maybe a bird’s eye view of this sector of Computer Science. I would like to hear from you, what do you think about it.

References:

Video:
1. What is Artificial Intelligence? In 5 minutes by Raj Ramesh
2. Understanding Artificial Intelligence and Its Future | Neil Nie | TEDxDeerfield
3. This Canadian Genius Created Modern AI ( Geoffrey Hinton )
4. But what is a Neural Network? | Deep learning, chapter 1
5. What REALLY is Data Science? Told by a Data Scientist by Joma Tech
6. Artificial Intelligence In 5 Minutes | What Is Artificial Intelligence? | AI Explained | Simplilearn
7. CogX 2018 — Symbolic Methods Coming Back

Article:
1. Data Engineer VS Data Scientist
2. The AI Hierarchy of Needs
3. A Beginner’s Guide to Data Engineering — Part I by Robert Chang
4. What is symbolic artificial intelligence?
5. Symbolic Reasoning (Symbolic AI) and Machine Learning

Don’t forget to give us your ? !

---

A brief introduction to AI was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

Via https://becominghuman.ai/a-brief-introduction-to-ai-22ca582493f6?source=rss—-5e5bef33608a—4

AI For Medical Diagnoses

AI has long been in existence, but it’s only in the last few years that we have seen some tremendous growth in this technology. The kind of potential that this technology holds for the future is incomparable. In the current scenario, we are witnessing a huge gap between the demand for Artificial intelligence developers this is primarily because most of the companies are now emphasizing on the need to have AI-driven technology. When we talk about the use cases of AI across different fields, then we cannot miss the healthcare segment where it is finding optimal usage.

Some worth a watch stat:

Since we will be exploring the avenues of AI in the field of healthcare, it becomes important to know how fast the industry is growing. As per the Grand View Research, the global AI in healthcare market size is going to rise with a CAGR of 41.5% and is expected to reach USD 31.3 billion by 2025. This rise in demand is accredited to various factors like the adoption of precision medicine, the growing importance of big data in healthcare, reducing the cost of hardware. Besides, the potential application of AI-based tools in healthcare has acted as a catalytic factor in making this technology gain so much momentum in recent times, and it is expected to grow in the times to come.

Some additional information:

1. In 2018, software solutions were the key uses of AI-based applications in the healthcare segment.
2. AI automation of clinical trial process was another key use of AI in the field of healthcare
3. North America was the place to showcase the highest adoption of AI, and it also had the largest revenue share
4. Asia Pacific region is expected to see the highest growth in AI and its based application. This is primarily because of the startups that are venturing in this field.
5. Companies like NVIDIA Corporation, IBM Corporation, Microsoft, Intel Corporation and DeepMind Technologies are a few of the companies that have shown tremendous growth in the field of AI.

All this data clearly indicates that we are going to witness a steep rise in the demand for AI-based applications and products. With the growing penetration of AI in the field of healthcare, we can expect the demand for artificial intelligence developers is also going to increase, and so is the demand for artificial intelligence training. Most of the companies motivate their employees to adopt Artificial Intelligence training. In fact, there is an organization that is now looking for AI certification as a key skill while hiring new candidates for a technical job.

Trending AI Articles:

1. AI for CFD: Intro (part 1)

2. Using Artificial Intelligence to detect COVID-19

3. Real vs Fake Tweet Detection using a BERT Transformer Model in few lines of code

4. Machine Learning System Design

There are several benefits which AI can bring to the healthcare segment, both the caregivers, hospitals and the patients. It enables personalized care, medication to the patients. Moreover, there are certain AI-based tools which are equipped to perform preliminary scanning in case of healthcare emergencies. These become extremely beneficial when there is a shortage of physicians and healthcare professionals.

Here is how AI can prove helpful in the healthcare segment:

When we talk about the varied use cases of AI in the field of healthcare, then you must be wondering, how will AI prove beneficial. Here we have enlisted a few of the many ways in which AI is finding applications in the healthcare field:

1. Virtual Assistants
2. Diagnosis
3. Robot-assisted surgery
4. Clinical Trials
5. Connected Machines
6. Fraud Detection
7. Cybersecurity
8. Dosage Error reduction

There are several ways in which AI can prove to be beneficial like they are less probably of errors, timely diagnosis, faster assistance etc. AI-based applications are pre-programmed, and with the assistance of technologies like machine learning, we can expect them to become better with time.

Here we would like to mention about telehealth which is the latest revolution of AI. We know that artificial intelligence is used to detect high-risk diseases on a larger scale and telehealth tools are used at homes of patients to prevent and treat health-risk situations. It reduced the need for readmission to the hospital. Such tools document different metrics, and in case of risk, the machine will automatically inform the practitioner about the same. This can prove beneficial by saving the life of the patient.

Conclusion

AI is a growing technology, and it is going to grow in the times to come; it has become one of the lucrative career options, and if you are planning to make a career in this field, then you must consider going for certification in artificial intelligence by the Global Tech Council. The platform will give you in-depth information about artificial intelligence and information related to the same. The certification in artificial intelligence training is rendered online, and it is best suited for both the freshers and working professionals. After successful completion of this course, you can explore opportunities in this field.

Don’t forget to give us your ? !

---

AI for Medical Diagnoses was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

Via https://becominghuman.ai/ai-for-medical-diagnoses-bf44dd86669?source=rss—-5e5bef33608a—4

The Gift of Feedback for Engineers

Remote working arrangements in light of Covid have amplified the effects of not being good at feedback for software engineers. Our relationships with each other are at the heart of how we move fast. When we don’t have the luxury to build our relationships in-person, feedback becomes paramount to ensuring we get our relationships right digitally.

It is a fundamental law of nature that in order to gain strength you have to push your limits, which is painful. This is easiest to see in gaining physical strength. Exercise is uncomfortable, it can be painful, but the more you endure the stronger you become. The mind is no different.

Another fundamental law of nature is that organisms will expend as little energy as possible to achieve their goals. Enduring pain, both physical and mental, consumes energy. If you want to grow you need to push hard to overcome this natural resistance.

Applied to Software Engineering:

• Don’t avoid confronting problems because they are painful to think about.
• At the start of each week identify the one problem you’re worried about most, and begin there.
• Regularly solicit constructive feedback from peers, reports (if applicable), and your manager.
• Work with your manager to understand expectations at the next career level, and try to consistently operate just outside of your comfort zone.
• At the end of each week ask yourself what you did during that week that you either failed at or were scared of doing. If nothing comes to mind you may not be pushing yourself enough, and in the long term may not live up to your full potential.
• Don’t endure more pain than you can tolerate. Exposure to pain should be sustainable. Avoid burnout.

Lesson: Accept reality

Objective reality and our individual perception of reality are different. Sometimes the difference is caused by ignorance, and sometimes it is intentional, to protect us. Either way, any difference will impede our ability to achieve goals. We make decisions based on a model of reality which isn’t true.

Recognize this delusion. Recognize that accepting reality, no matter how painful, will make us better decision makers. Adopt strategies for countering this human weakness, to get closer to what’s true, and incorporate them into your decision making process.

Applied to Software Engineering:

• When attempting to acquire new knowledge, apply the scientific method.
• Establish a mentorship relationship with someone who is more experienced in a particular domain than you are. Regularly ask this mentor how they interpret various scenarios in this domain, and contrast their interpretation with your own.
• Accept that you will never be perfect, that you have weaknesses, but that you also have strengths. Understand what those strengths are and play to them.
• Where possible make decisions based on data, but only if you understand and are satisfied with the source of the data.
• When reality doesn’t turn out the way you want, fall back to your personal principles for reassurance and guidance.
• Ask multiple, credible people the same question to test the believability of the answer.

Lesson: Reflect after pain

Once emotions subside, take time to explore and examine what happened. Sometimes this has to happen in a group setting, but it’s most effective when done alone, when you’re free from the judgements of others and can be honest with yourself. During reflection, identify what adjustments you need to make to your mental model of reality, to avoid a recurrence of the pain. It may be useful to make a plan, to ensure the change is truly internalized through repetition.

Applied to Software Engineering:

• Once a project is complete, host a retrospective with your team to identify lessons learnt. This is particularly important following a failed project.
• Find alone time to reflect on constructive feedback shared with you, to internalize it and adjust your behavior accordingly.

Why don’t we give feedback?

Many of us, as software engineers don’t like to give feedback. Let’s break those beliefs down and call on you to change your view on feedback:

1. Feedback takes so much effort

Feedback can be effortful if you do not have the muscle for it, and if you’ve left a pile of it festering and haven’t been delivering feedback regularly. Yet, this is a situation where you may need to slow down to move fast in the long run. This is exactly why we need to start practicing giving and receiving feedback today.

Trending AI Articles:

1. AI for CFD: Intro (part 1)

2. Using Artificial Intelligence to detect COVID-19

3. Real vs Fake Tweet Detection using a BERT Transformer Model in few lines of code

4. Machine Learning System Design

Think of feedback that you need to give as a small acute pain. Sure, it might be fine that you have one small acute pain indefinitely into the future. Yet, if you have one in your knee and elbow and stomach, even though individually these pains are small, in sum they affect your physical and mental functioning in the long term. We should not accept living with these “small” feedback “pains” because they add up and hurt all of us in the long run.

Your feedback can also directly affect other people. When you have feedback for someone, sometimes, other people may have similar feedback for the same person. The person to whom you need to give feedback may not be aware that multiple people have feedback for him or her because no one has said anything. By giving feedback, you may be scaling your impact by helping identify behaviors that have impact on multiple people and teams; in sum, your effort doesn’t just help you, it helps others.

2. Feedback leads to awkward conversations

If you are not regularly having some honest, awkward conversations, you may be leaving problems unsolved. “Because if it’s not a bit awkward, you’re not talking about the real stuff.”

Feedback is not always easy, and we need to acknowledge that. What is good for us isn’t always easy to do, but our fear of having these conversations should not paralyze us into inaction; if we are fearful, we need to find ways to tackle that fear head on:

Are you afraid that your feedback will not be well-received?

• Learn how to give feedback effectively: There are numerous online courses that give you the tools to deliver feedback well, even in high-stakes situations. Equip yourself with the tools to give feedback well in all situations.
• Prepare: Give feedback having thought about it and having prepared specific examples (facts) — go as far as writing them down because it will help you focus on what is factual and you have something to refer back to if you feel nervous in a meeting.
• Practice: Role-play with colleagues and friends in preparation for tough feedback conversations, and it could helped you hone your tone and delivery and anticipate questions. If you practice presenting for important meetings, then you can also practice to give effective feedback.

Are you afraid of repercussions of giving feedback?

• If you feel uncomfortable giving feedback, you can speak to your manager, or the person’s manager to give the feedback privately and understand how best to channel this feedback. This should be a last resort since you should try to resolve and give feedback to the person directly, but this may not be possible in all situations. The idea is that, especially if you feel uncomfortable, this doesn’t mean that your feedback should not be heard.
• If the feedback is about your manager, you should consider giving the feedback to your skip level (manager’s manager).

Are you afraid that you’ll damage the relationship by giving feedback?

• ‘The more we give and receive feedback, the easier it becomes. Most feedback isn’t hard to hear because it’s right or wrong, but because we have an emotional response to the idea of “feedback.”’ It may be hard to have the conversation, but especially if it’s a relationship you care about, having this conversation gives it a chance to advance.
• “Remember that feedback is a two-way conversation. Building a healthy company requires approaching our work with an open mind, honest dialogue, and true desire to continue learning and growing, so ultimately, feedback should strengthen our relationships.”

Practically speaking, you can share context with your manager and ensure you and your manager are aligned on the feedback approach. Your manager can give further advice and help moderate or mediate the relationship, should it be required.

3. Feedback means I’m saying something bad about someone or getting them in trouble

Your first path of feedback should always be to give the feedback to the individual directly (where possible), and as quickly as possible after specific situations which are the basis of your feedback. In most situations, feedback should not be a surprise. The point of feedback is to have a discussion and find a path forward together.

Feedback is not limited to constructive feedback; it can be positive recognition. We often don’t stop to think and recognize things that we appreciate from each other because we think “this person already knows that they do a great job!”. This is often not the case, and positive recognition can be just as important as the areas we look to improve. It helps us celebrate success and strengthen our relationships with each other.

Feedback is not absolute. It is a two-way conversation. You’ll be surprised to learn that in a lot of situations, the person to whom you’re giving the feedback was not aware of how their actions may have made other people feel. And so, feedback should be viewed as a way of saying “Hey, remember when this happened and you did X? You may not be aware, but it felt like Y.” Feedback does not make judgments of right or wrong on an observed behavior, but it shares with the actor the receiver’s interpretation of events. At the end of the day, feedback is just good communication and should be viewed as an opportunity to improve a working relationship. But sometimes, it doesn’t always improve a relationship, in which case you might think…

4. Feedback is a waste of time because the person doesn’t change their behaviour

Feedback is never a waste of time. If there is genuinely something problematic happening and it’s a pattern of behavior, your feedback is a means of holding someone accountable to their behavior by first attempting to communicate with the person to find a better path forward. If someone had a pattern of behavior that was detrimental to the company and everyone thought “there’s no point in giving feedback”, not giving feedback would mean that this behavior would continue to go unaddressed and you, your team, and the company would be negatively affected. If someone isn’t responding to feedback, working with your manager to find appropriate paths of escalation is now possible because you’ve made the effort to repeatedly give feedback to this person in the first place.

Conclusion

If we care about having impact as a software engineer at your company, we should also care about the relationships with the people on our teams, because these relationships elevate our mutual impact. Relationships are at the heart of everything we do; feedback helps make those relationships high-functioning.

When done well, feedback helps us grow and move fast in the long run, as individuals, and as a company. Take some time today to prepare and give or receive some feedback for someone with whom you work. Feedback is truly a gift.

Don’t forget to give us your ? !

---

The Gift of Feedback for Engineering was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

Via https://becominghuman.ai/the-gift-of-feedback-for-engineering-1ce612b14928?source=rss—-5e5bef33608a—4

This article focuses on design principles of the PCA algorithm for dimensionality reduction and its implementation in Python from scratch.

Originally from KDnuggets https://ift.tt/2yqHD2C

There’s no free lunch in machine learning. So, determining which algorithm to use depends on many factors from the type of problem at hand to the type of output you are looking for. This guide offers several considerations to review when exploring the right ML approach for your dataset.

Originally from KDnuggets https://ift.tt/2TsMXtv

In this article, I’ll introduce you to a hot-topic in financial services and describe how a leading data provider is using data science and NLP to streamline how they find insights in unstructured data.

Originally from KDnuggets https://ift.tt/3dYdsPo

Pandas is instantly familiar to anyone who’s used spreadsheet software, whether that’s Google Sheets or good old Excel. It’s got columns, it’s got grids, it’s got rows; but pandas is far more powerful. Save 40% with code nlkdpandas40 on this book, and other Manning books and videos.

Originally from KDnuggets https://ift.tt/36fBL8V