365 Data Science

AI’s transformative potential has been the prime mover for its widespread adoption among organizations across the globe and continues to be the utmost priority for business leaders. PwC’s research estimates that AI could contribute $15.7 trillion to the global economy by 2030, as a result of productivity gains and increased consumer demand driven by AI-enhanced products and services.

While artificial intelligence (AI) is quickly gaining ground as a powerful tool to reduce costs, automate workflows and improve revenues, deploying AI requires meticulous management to prevent unintentional ramifications. Beyond the compliance to the laws, CEOs bear a great onus to ensure a responsible and ethical use of AI systems. With the advent of powerful AI, there has been a great deal of concern and skepticism regarding how AI systems can be aligned with human ethics and integrated with softwares, when moral codes vary with culture.

Creating responsible AI is imperative to organizations and instilling responsibility in a technology requires following criteria to be fulfilled —

• It should comply with all the regulations and operate on ethical grounds
• AI needs to be reinforced by end-to-end governance
• It should be supported by performance pillars that address subjects like bias and fairness, interpretability and explainability, and robustness and security.

Key Dimensions For Responsible Ai

1. Guidance On Definitions And Metrics To Evaluate AI For Bias And Fairness

Value statements often lack proper definitions of concepts like bias and fairness in the context of AI. While it’s possible to design AI to be fair and in line with an organisation’s corporate code of ethics, leaders should lead their organizations towards establishing metrics that align their AI initiatives with company’s values and goals. CEOs should comprehensively lay down company’s goals and values in the context of AI and encourage collaboration across the organization in defining AI fairness . Some examples of metrics include:

• Disparate Impact: The ratio in the probability of favorable outcomes between the unprivileged and privileged groups.
• Equal Opportunity Difference: The ratio of true positive rates between the unprivileged and privileged groups
• Statistical Parity Difference: The difference of the rate of favourable outcomes received by unprivileged group and the privileged group
• Average Odds Difference: The average difference of false positives and true positives between unprivileged group and the privileged group
• Theil Index: The inequality of benefit allocation for individuals

2. Governance

Building Responsible AI practices in the organization requires the leadership to build proper communication channels, cultivate a culture of responsibility, and build internal governance processes that match with the needed regulations and industry’s best practices.

Trending AI Articles:

1. How to automatically deskew (straighten) a text image using OpenCV

2. Explanation of YOLO V4 a one stage detector

3. 5 Best Artificial Intelligence Online Courses for Beginners in 2020

4. A Non Mathematical guide to the mathematics behind Machine Learning

End-to-end enterprise governance is extremely critical for Responsible AI. Organizations should be able to answer the following questions w.r.t AI initiatives:

1. Who takes accountability and responsibility
2. How can we align AI with our business strategy
3. Which processes can be optimized and improved
4. What are the essential controls to monitor performance and identify problems

3. Hierarchy Of Company Values

AI development isn’t devoid of trade-offs, in fact, while developing AI models there is often a perceived trade-off between the accuracy of an algorithm and the transparency of its decision making i.e, how explainable its predictions are for stakeholders. A high accuracy AI model can lead to the creation of “black box” algorithms, which makes it difficult to rationalize the decision making process of the AI system.

Likewise, trade-off exists while training AI. As AI models get more accurate with more data, gathering a large volume of data itself can increase privacy concerns. Formulating thorough guidelines and hierarchy of values is vital to shape responsible AI practices during model-development.

4. Security And Reliance

Resilience, security and safety are essential elements of AI for it to be effective and reliable.

• Resilience: Next-generation AI systems are going to be increasingly “self-aware,” with a capability to evaluate unethical decisions and correct faults.
• Security: AI systems and development processes should be protected against potential fatal incidents like AI data theft, breaches in security that lead to systems being compromised or “hijacked”.
• Safety: Ensure AI systems are safe to use for the people whose are either directly impacted by them or will be potentially affected by AI-enabled decisions. Safe AI is critical in areas related to healthcare, connected workforce, manufacturing applications etc.

5. Monitoring AI

AI needs to be closely monitored with human supervision and its performance should be audited against key metrics like accountability, bias, and cybersecurity. A diligent evaluation is needed as biases can be subtle and hard to discern and a feedback loop needs to be devised to effectively govern AI and fine tune biases.

A continuous monitoring of AI will ensure that the models will recreate an accurate real-world performance and take user feedback into account. While issues are bound to occur, it is necessary to adopt a strategy that comprises short-term simple fixes and longer-term learned solutions to address the issues. Prior to deploying an AI model, it is essential to analyze the differences and understand how the update will affect the overall system quality and user experience.

6. Explainability

Explainable AI (XAI) is defined as systems with the ability to explain their rationale for decisions, characterize the strengths and weaknesses of their decision-making process, and convey an understanding of how they will behave in the future. While it’s desirable to have complex models that perform exceedingly well, it would be erroneous to assume that the benefits derived from the model outweigh its lack of explainability.

AI’s Explainability is a major factor to ensure compliance with regulations, manage public expectations, establish trust and accelerate adoption. And it offers domain experts, frontline workers, and data scientists a means to eliminate potential biases well before models are deployed. To ensure that model outputs are precisely explainable, data-science teams should clearly establish the types of models that are used.

7. Privacy

Machine Learning models essentially need large sets of data to train and work accurately. A caveat to this process is that a lot of data can be sensitive. It is essential to address the potential privacy implications in using such data. This demands enterprises to adhere to legal and regulatory requirements, be sensitive to social norms and individual expectations, and finally have adequate transparency and control of their data.

Conclusion

A Responsible AI framework enables organizations to build trust with both employees and customers. In doing so, employees will rest their faith in the insights delivered by AI, willingly use it in their operations and ideate new ways to leverage AI in creating greater value.

Building trust with customers, opens the floodgates to use consumer data that can be used to continually improve AI and consumers will be more willing to use your AI-infused products because of the trust in the product and the organization. This also improves brand reputation, allows organizations to innovate, compete and most importantly, enables society to benefit from the power of AI than be paranoid about the technology. via Acuvate.com

Don’t forget to give us your ? !

Responsible AI Practices Your Organizations Should Follow For Better Trust was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

Via https://becominghuman.ai/responsible-ai-practices-your-organizations-should-follow-for-better-trust-eba438898e0d?source=rss—-5e5bef33608a—4

Accessing and keeping constant track of business KPIs and reports is hard. CxOs, VPs, managers and other senior management leaders need to continuously monitor different types of KPIs relevant to their role. But in order to access them, they have to undergo the tedious process of opening their Power BI app (or any Business Intelligence application for that matter), navigating through dashboards, and filtering based on the parameters.

This is a difficult workflow which doesn’t fit in the busy schedules of these leaders. Remember, the average employee switches between 35 job-critical applications more than 1,100 times every day.

Companies have developed loose workarounds to overcome this. In fact, during our conversations with large enterprise customers, we’ve found that many of them have deployed a dedicated person whose sole job is to take screenshots of the analytics reports in the Business Intelligence (BI) app and send them to senior managers via emails or publish them on intranets.

But, as you can probably guess, these static images won’t be either timely or interactive. By the time the report reaches a CxO, the data in it may not be up-to-date. And if a user needs deeper insights about a report, they again need to request and wait for a different screenshot.

This lack of timely access to relevant data results in poor business decisions and lost opportunities.

Trending AI Articles:

1. How to automatically deskew (straighten) a text image using OpenCV

2. Explanation of YOLO V4 a one stage detector

3. 5 Best Artificial Intelligence Online Courses for Beginners in 2020

4. A Non Mathematical guide to the mathematics behind Machine Learning

Business Intelligence And Analytics Workspace In A Modern Intranet.

To overcome the above-mentioned challenges, we’d like to present the possibility of surfacing BI and analytics dashboards within the home page of an intranet.

Modern SharePoint intranets like Mesh 3.0 are acting as a central hub of information for several organizations. With their flexible integrations with enterprise apps, intranets are enabling employees to get their work done with a unified interface of communication, collaboration and information sharing.

To take this innovation to the next level, we’re introducing the BI & analytics workspace capability in Mesh 3.0. By integrating the SharePoint intranet solution with your BI app like PowerBI, Tableau etc. you can access the live interactive analytics dashboards and reports within the intranet

How Does The Integration Of Business Analytics Apps With An Intranet Work?

Upon the intranet integration with a BI app, users can:

• Subscribe to their desired BI dashboards and reports which will then surface on their intranet home page
• Access interactive and the most up-to-date dashboards which can be drilled down
• Click and open the dashboard in the respective BI app for further insights
• Set and get personalized alerts on the needed KPIs either on the intranet home page or via an intranet chatbot
• Get a personalized data experience: The intranet tailors the dashboards based on users’ role, department, location, interest etc.

Reduce The Time To Access Business KPIs

By retrieving the needed BI reports within an intranet, business users can experience the following benefits:

1. Reduce the time to find information on KPIs
2. Have easy and quick access to relevant dashboards
3. Improve decision making by getting the latest data and interactive reports to the fingertips
4. Increase productivity by retiring workarounds like using a middleman to send reports’ screenshots
5. Stay updated on changes in KPIs

Productivity Analytics In The Intranet

In addition to business analytics, Mesh 3.0 also features a personal productivity analytics capability which lets knowledge workers to track their work patterns and understand the time spent on different activities. Intranets can provide insights into how users are spending their time by analyzing

• The usage data of the intranet
• The usage data of productivity, communication and collaboration apps (Outlook, Office 365, SAP, Teams, etc.)
• Enterprise search activity data

These reports provide productivity insights such as time spent on different projects, activities, meetings, collaboration with peers, etc. They also provide helpful suggestions on improving users’ productivity and work life.

Explore Mesh — An Autonomous SharePoint Intranet

Mesh 3.0 — world’s first autonomous intranet, features many more innovative capabilities like:

1. Cognitive enterprise search
2. AI-powered innovation management
3. Azure knowledge mining
4. Personalized content experiences based on users’ role, activities, interests, geo etc.
5. Intranet chatbots
6. Flexible integrations with 3rd party apps
7. Collaboration and communication tools

Source

Don’t forget to give us your ? !

Business Intelligence Dashboards Within An Intranet was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

Via https://becominghuman.ai/business-intelligence-dashboards-within-an-intranet-4ef155f04075?source=rss—-5e5bef33608a—4

Time Series and How to Detect Anomalies in Them — Part III

Eventually easier than it seemed

Hello there, my name is Artur.

You might be reading this intro for the third time — and if this is the case, I appreciate your sticking with this article series.

I am the head of the Machine Learning team in Akvelon-Kazan and you are about to read the last part of the tutorial for anomaly detection in time series.

During our research, we’ve managed to gather a lot of information from tiny useful pieces all over the internet and we don’t want this knowledge to be lost so we are sharing it with you!

We already dove into the theory and data preparation in Part I and defined and trained three models in Part II:

• Part I — Intro to Anomaly Detection and Data Preparation
• Part II — Implementation of ARIMA, CNN, and LSTM

We reuse our code so if something seems unclear, consider visiting the previous parts once more.

Fantastic, let’s complete this series!

Just to briefly remind the tools that we use:

• Jupyter Notebooks environment for the implementation of the models
• Scikit-learn for some data preprocessing
• Statsmodel library for ARIMA model
• PyTorch for neural networks
• Plotly for plots and graphs

And what type of models and how we trained:

1. ARIMA statistical model — predicts next value
2. Convolutional Neural Network — predicts next value
3. Long Short-Term Memory Neural Network — reconstructs current value

Anomaly Detection with Static and Dynamic Threshold

Amazing, we trained all three models! But every line of the code before was just the preparation for the anomaly detection.

So just after a very small amount of additional preparations, we will be able to finally detect anomalies.

What exactly lies behind these “additional preparations”? These things (remember the “Saying what we want from our models out loud” from Part I?):

1. Calculation of the errors for each item in datasets
2. Threshold calculation based on errors

And then we will be able to detect anomalies extremely fast like literally just comparing errors with the threshold.

What are we waiting for? We’re ready for this!

The error calculation differs for each model due to different implementations of Datasets. The algorithm stays the same.

ARIMA’s errors calculation:

Just as a reminder — we use absolute error for ARIMA because it causes better results. And if you are wondering how we came to this, the answer is — we just tried and it worked.

CNN’s errors calculation:

Trending AI Articles:

1. How to automatically deskew (straighten) a text image using OpenCV

2. Explanation of YOLO V4 a one stage detector

3. 5 Best Artificial Intelligence Online Courses for Beginners in 2020

4. A Non Mathematical guide to the mathematics behind Machine Learning

LSTM’s errors calculation:

Threshold calculation — common for all three models:

Static threshold

This threshold is calculated with this formula of the three-sigma rule.

Dynamic threshold

For the dynamic threshold, we will need two more parameters — window inside which we will calculate threshold and std_coef that we will use instead of 3 from the static threshold formula.

• For ARIMA window=40 and std_coef=5
• For CNN and LSTM window=40 and std_coef=6

These two parameters are empirically chosen for each model using only the training data.

You may wonder — “Why does he always emphasize the usage of only training data? Why can’t I also use validation to choose better parameters?”.
The reason why we use just training data for choosing the parameters of our models is that this is the only way we can be sure that our models will work on data from the real world outside the training dataset. The validation part of the dataset imitates such real-world data and provides a better understanding of models’ capabilities because we know — it wasn’t used to train our models.

Let’s get down to business! Here is the code to calculate the dynamic threshold:

And the last element to fulfill our puzzle is metrics calculation. What kind of metrics? I am glad you asked. We calculate every base metric to fully analyze the models’ performance:

• Confusion matrix to see how a model performs in detail
• Precision to see how precisely our model predicts
• Recall to see how a model detects true anomalies
• F2-score to see the combined precision and recall, we are using F2 instead of F1 because detection of true anomalies is more important than avoiding false anomalies (recall is more important than precision)

Excellent! We can move to the piece of code with exact anomaly detection.

ARIMA with static threshold:

For each model, we are going to filter errors with the given threshold and then simply return indexes of unfiltered ones. These unfiltered values we will consider as detected anomalies!

And of course, we are going to visualize everything that we detected! (By still using the same code from the Part I)

We will leave the metrics until the results part. But here are the code and printed confusion matrices:

Yeah, seems not so good (because of many falsely detected anomalies), but it still catches every anomaly.

ARIMA with dynamic threshold:

Let’s do the same for the dynamic threshold and see if it can change the situation.

The code for metrics is the same, so we can skip it and take a look at the confusion matrices.

Well, these look much better (no more huge amount of incorrectly detected anomalies)! A tough baseline for our neural nets!

NN’s anomaly detection

For both neural nets, we will provide a unified generic function for anomaly detection.

And that’s it! We can effortlessly process the results of neural nets.

CNN with static threshold:

It seems that our CNN model overfitted — it has an enormous amount of incorrect anomalies — but there is no need to make hasty decisions, it is better to look onto results with the dynamic threshold.

CNN with dynamic threshold:

And let’s do the same with the dynamic threshold:

The metrics calculation is still the same.

These results are better than ARIMA’s. We already can say that we didn’t waste time on this!

And the last model (but certainly not the least) is LSTM.

LSTM with static threshold:

Once again, metrics calculations are identical to CNN’s.

Here we have the same situation as with CNN, but now we know that the dynamic threshold will reveal the truth!

LSTM with dynamic threshold:

And the dynamic evaluation certainly made a near-perfect detector out of our LSTM model.

Real-time evaluation with static/dynamic threshold

If it is hard to figure out from the code how to use these models in real-life data (and this is normal), so here are some visualizations of the real-time evaluation:

The top chart shows the original data with true anomalies and detected anomalies. On the bottom chart, we can see the error of a model with the purple static threshold line.

And here is the visualization of the same process with the dynamic threshold.

As you can see, the dynamic threshold adapts to the dispersion of the error. That is why the threshold is low when the error deviates a bit and high otherwise.

Results of the models

Finally, we can compare the metrics to be sure that we correctly put the LSTM onto the first place. We are using F2-score to decide which model is the best. Precision and recall are shown separated for the understanding of weak and strong sides of our models.

However, ARIMA performs slightly better with the static threshold, and the neural networks outperforms it with dynamic threshold — especially LSTM.

Ultimate Conclusion

Lastly, I would like to emphasize that these models can already be taken for production with not so much effort.

Nevertheless, these models are far from their limits and can be enhanced via:

1. Increasing the amount of training data
2. Adding other metrics such as memory, network, etc
3. Combination of LSTM and CNN architectures
4. Feature Engineering

Thank you very much for your attention, I hope that this tutorial gave you some understanding and hints on implementation.

And don’t stop looking for anomalies!

About us

We at Akvelon Inc love cutting edge technologies in mobile development, blockchain, big data, machine learning, artificial intelligence, computer vision, and many others. This article is the result of one of many projects developed in our Office Strategy Labs where we’re testing new technologies and approaches before delivering them to our clients.

If you would like to work with our strong Akvelon team — please see our open positions.

Designed and implemented with love in Akvelon:
Team Lead — Artur Khanin
Delivery Manager — Sergei Volynkin
Technical Account Manager — Max Kostin
ML Engineers — Irina Nikolaeva, Rustem Saitgareev

Time Series and How to Detect Anomalies in Them — Part III was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

Via https://becominghuman.ai/time-series-and-how-to-detect-anomalies-in-them-part-iii-f72e800e15e2?source=rss—-5e5bef33608a—4

There are so many new and exciting businesses being created all the time. They are also in competition with each other to ensure that they offer the best products or services possible. With this high level of competition comes new solutions to manage their businesses. Only the best businesses which work efficiently, have high engagement with their customers, and create innovative products/services can become highly successful businesses. There have been many advancements in the digital world that can help businesses be successful. One of those advancements is conversational AI, including chatbots. This article will inform you of the benefits of implementing AI into your business as well as other digital tools, such as Google Digital Garage, that you can use.

What is Conversational AI?

Before we can discuss the many benefits, we must first understand what conversational AI is.

It is the process of creating technology on computers to mimic human interaction. It is used for many different things, such as chatbots, phone calls, emails, and social media. This allows customers to ask a question, and the AI will give a response as a real person would. Chatbots are usually used for questions that can provide fast answers. For example, asking a business about returns policies, resetting passwords, and other simple questions.

On the other hand, a more advanced conversational AI is virtual assistants such as Google Home, Amazon Alexa, Apple Siri, and more. These assistants are constantly developing, increasing their skills network, and can also be used for businesses as well as personal use.

Virtual customer assistants perform advanced tasks for customers, and there will be more innovative AI tools to bring to businesses in the future. They can provide answers to complex problems and process all the information provided to them to find a solution.

Conversational AI also learns from previous interactions. If you tell the AI that something, they said was wrong, they are designed to learn from this and not do the same thing again. When in a business environment it is important that they give accurate information so this is a tool that will be developed even further.

Trending AI Articles:

1. How to automatically deskew (straighten) a text image using OpenCV

2. Explanation of YOLO V4 a one stage detector

3. 5 Best Artificial Intelligence Online Courses for Beginners in 2020

4. A Non Mathematical guide to the mathematics behind Machine Learning

The Benefits

1. It Can Reduce the Need for Customer Relation Staff

One of the benefits of conversational AI on the management of businesses is that it can reduce the number of employees that the business has to employ on some of the more repetitive tasks such as simple questions. In turn, this reduces company costs. Of course, it is important to still have some employees as customer relations employees so that customers can talk to humans when they have complex questions, or they want to feel more connected to the company.

However, the reduced need for staff in this area can encourage the rest of the staff to develop their management skills in other areas. For example, becoming a part of the CMI and developing leadership skills.

2. Provide Around the Clock Support

Another benefit is that if you have a global organisation, you will be able to provide customer service for your business at all hours. This is especially useful for businesses that would need to support their customers at whatever time in the day. One example of a business would be an internet provider. There are only so many hours of customer support that a business can provide if they are based in one country.

3. Customers Get Timely Responses

Many customers prefer to use automated response software for their queries. This is because they can get answers quickly and would not have to wait for a member of staff to get in contact with them. If the business is particularly busy, then it would be more ideal for a customer to use the conversational AI software. Most of the responses on a chatbot and those alike would provide instant responses and be able to solve any small problems.

4. AI Can Help Staff with Prompts

For a lot of conversational AI that has been created for answering the questions of customers that are more complex than simple questions and answers, businesses often use a monitored system. This means that much of the conversation is led by the computer and the staff only steps in when more help is needed. This is also shown to be useful because the computer can store a lot more information than a human can. It can then interpret what the customer says and find an answer from millions of potential answers. For example, if someone asked a question about a company’s privacy policy, but the member of staff was unsure or would take a while to research the answer, the computer would be able to provide this information almost instantly. This increases the number of tasks that can be done in a day and increases the productivity of the business overall.

Other Digital Tools That Help Businesses

Aside from conversational AI which we can expect to see expand over the coming years, businesses also benefit from a range of other digital tools.

1. Process Automation

Artificial intelligence can also be used for other purposes in a business. Some processes that are carried out by computers include sifting through job applications for recruitment processes, emailing, and personalised marketing.

2. Google Digital Garage

Another digital tool commonly used for helping the management of businesses and helping them to thrive is Google Digital Garage. This has many different tools from marketing techniques to e-commerce tools and social media tools. Digital marketing is one of the most useful tools to attract more customers. Digital marketing can also be incorporated into the AI world as it can help a business focus their marketing efforts on a particular target audience and discover which customers would be more likely to choose the business’s products/services.

3. Project Management Software

Many businesses use project management software because it can help teams keep up with their workload and help projects run smoothly. It is also well known that project management is becoming increasingly overtaken by AI. It is thought that AI will be able to take over the role of a project manager in the future. AI can track the performance of employees and automate tasks to get the project finished by a particular date.

Final Thoughts

This article has informed you about how AI can benefit businesses in a variety of different ways. One of the most beneficial parts of using AI is that it can reduce business costs and time which is essential for business growth long term. Generally, AI is a tool that can help many businesses increase their productivity.

There is a lot more that we should expect to see from AI over the coming years. Things such as AI phone calls which can call up businesses and arrange things like hair appointments have already been developed and it is likely we would see something similar that businesses have with customers and clients.

The reality is that AI can work much faster than humans can. They also do not get tired, so this means that they work much better without having to take shifts. Finally, they can process a lot more information than we can, and we may see more of the repetitive tasks in businesses being taken up by computers.

Benefits of Using a Conversational AI to Efficiently Manage Your Business was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

Via https://becominghuman.ai/benefits-of-using-a-conversational-ai-to-efficiently-manage-your-business-32ff9cc8b426?source=rss—-5e5bef33608a—4

Via https://becominghuman.ai/how-to-build-lean-ai-startups-including-real-world-case-studies-84306075a372?source=rss—-5e5bef33608a—4