365 Data Science is an online educational career website that offers the incredible opportunity to find your way into the data science world no matter your previous knowledge and experience.
We are surrounded by a lot of gadgets and applications based on Artificial Intelligence. A lot of people a very much dependent on the same and AI has slowly become a part of people’s lives. Artificial intelligence algorithms are used for making decisions as well as assisting us in various aspects of everyday life. Where should we start? From Alexa, Siri, Google assistant to Tesla, Boxever, Netflix, the world is slowly getting adapted to AI.
CRM refers to a Customer Relationship Management system.CRM providers typically provide software services that will enable the organization to keep a track of customer data.
It also helps to use the data accordingly and provide better service to the customer. Hence, strengthening the relationship between the organization and the customer. CRM can be used for various purposes including sales and customer service. When the representatives and sales team will have the information of customers in handy, they will be able to understand and analyze the relationship between the Microsoft dynamics CRM company and the customer.
Big Data Jobs
This will be a great help for the sales team to find leads and efficiently sell their products. Nowadays many CRM providers let their customers access the software from anywhere through the cloud. This will provide the comfort of mobility as well as reduces the total cost of managing and handling the software.
Many businesses are incorporating AI with CRM to make certain areas of customer management easy and much efficient.
What can simplify our everyday tasks more than a virtual assistant, it is not deniable when I say that many of us are adhered to using virtual assistants for doing a lot of things every day. Many companies provide a virtual assistant to help the users understand the application and solve simple queries and attend to their problems.
This is an excellent way to communicate with the customers without requiring a lot of man force. These Virtual assistants can also be a great way to assist the customer and improve their work efficiency by doing simple tasks.
By recording and analyzing calls and contact history:
Artificial intelligence can be generally used to keep a record of all sales calls as well as client meetings and keep aside some details that can be further analyzed or which have important details.
By splitting and grouping customer info:
The choices and tastes of customers will always be different. Sometimes the requirements and choices of customers can vary based on different aspects. For example, customers of a certain location may need certain services. This can be identified and easily kept a track of using Artificial intelligence
Attending customer calls manually on a big scale is a time-consuming process. Moreover, as the business grows the requirement of representatives attending and helping customers will grow.
Hiring a lot of people will also be an expensive task. This is why a lot of companies and organizations are automating their call center assistance. Mostly AI is used only to perform simpler tasks and understand and note the caller complaint and take small inputs.
Predicts leads by analyzing customer pattern:
Artificial intelligence is also used for predicting leads and purchases of a product based on the purchase history of previous clients or customers. This will help in the prediction of the company’s annual turnover and revenue. Which will in turn help in the company’s growth.
How to prepare your CRM for the Incorporation of AI?
To add AI to your Customer relationship management software, the first thing required is a good AI algorithm, every task that we would want to automate and every function we would like the software to do must be added to the algorithm. Now, this algorithm must be tested with various inputs before adding it to the final software.
The second but equally important step is to collect data, clean it and provide it to the CRM which will further be the input for the software algorithm. Whereas in some of the CRM applications Artificial intelligence algorithms are inbuilt. For instance Dynamics 365 analyses customer data and produces metrics and actionable data.
How to acquire and clean and process data?
Data acquisition is extremely important as it is the input of many AI algorithms. Although we can transfer data from various applications to CRM, many CRM providers facilitate us by integrating their other applications from where the data can be taken and utilized.
In Dynamics 365, we will be able to import and export data very conveniently from all Microsoft applications like MS word, MS Excel, PowerPoint. This will make data acquisition quite simple.
After getting the required data we should clean the data, remove noisy and excess data, and remove duplicates. After cleaning the data we should segment and organize it. Now, this data can be provided as input to the AI algorithm.
Before using the system, the AI should be properly trained and the ultimate target of the organization must be specified to the system. Also, the system should be trained to recognize and act accordingly when any event happens.
CRM and AI in various Fields:
Artificial intelligence and CRM in sales:
AI can be used for various aspects of sales. Firstly, It can be used whether or not to consider a proposal. It will also help to determine if the lead is a bad or a good match. Whether to pursue a lead or not. Artificial intelligence is also used to transcribe calls and meetings and analyze risk factors and help us make decisions.
Artificial intelligence and CRM in Marketing:
Various marketing strategies can be automated. Marketing campaigns such as email marketing and social media marketing can be automated using AI. New products can be suggested to customers who purchased earlier. This can be done by keeping a track of their purchase history and analyzing it to find products that match their choice.
Artificial intelligence and CRM in customer service:
As mentioned earlier, the responses for service calls can be automated, and following the complaint or request the required data can be accessed. The addition of Artificial Intelligence to your CRM will be extremely beneficial to any business or organization.
Artificial intelligence is getting smarter by the day. Today, powerful machine learning algorithms are within reach of normal businesses, and algorithms requiring processing power that would once have been reserved for massive mainframes can now be deployed on affordable cloud servers. Natural language processing of the kind seen in popular chatbots may appear mundane, but it wasn’t all that long ago it was the stuff of science fiction.
You Need AI in Your Business
Gartner ranks augmented data management, NLP and conversation AI as some of the key coming trends for data and analytics. Data annotation is an important part of supporting AI to perform those tasks well. If you’re not putting good data into your models, you won’t get smart responses out. According to Gartner, up to 85% of AI projects will deliver erroneous results by 2022 due to biases in their training data.
Data mining and annotation skills are essential, yet53% of organizationssay that their own data mining skills are “limited”.
Big Data Jobs
What is Data Annotation?
Data annotation is a crucial part of making your AI smarter. It involves labeling the data that you feed to your machine learning algorithms so that the algorithms can learn to process the information that they see correctly. Data annotation is a painstaking process that can involve adding precise and mundane (to humans) notes to thousands upon thousands of images, pieces of text, or other data.
For data annotation to be done correctly, it is important that the humans who are working on the training data understand the scope of the project and what the algorithm is looking for. Having trained teamwork on the project, or at least explain to your in-house team what is required for data labeling, can help to maximise the efficiency of the project.
What Needs to Be Annotated?
Exactly what will need to be annotated depends on the type of project that you’re working on. A deep learning algorithm would need different inputs to a conversational AI or chatbot.
Data annotation takes time. According to a survey conducted by Algorithmia, 40% of companies report that it takes more than a month to deploy a machine learning model into production, and 81 percent of companies say that the training process is more difficult than they thought it would be.
It can be tempting to handle data annotation within your business, however, this is a waste of your in-house data scientist time. Data scientists reportedly spend just 20% of their time on analysis, with the bulk of their work being sanitizing and processing data. Outsourcing your data annotation will give your project a chance to get off the ground, and free up your data scientists to focus on their core skills.
Some basic annotations can be “crowdsourced”, and this is an affordable way of getting a significant number of annotations done. If your algorithm requires more than simple sentiment data or descriptions of mundane pictures, then your annotation team may need more detailed training. Some annotations require input from subject matter experts. This is particularly true in engineering, legal, scientific or medical fields. If your machine learning algorithm is going to be creating predictions or responding in mission-critical situations it is vital that the model is given accurate inputs. Only a subject matter expert can train an AI in a complex subject.
Choosing Your Annotation Vendor
If you are looking for assistance to train your AI, consider the following:
How much data do you need them to work through?
How diverse a data-set do you need?
Is the data sensitive?
If the data is sensitive, what precautions do you want the team to take?
How quickly do you need the annotations to be done?
How important is accuracy?
Do the annotators need specific knowledge?
Once you have your data annotation wishlist, you can start the process of choosing a vendor.
Write a Statement of Work: A statement of work defines the expectations that you have of the data annotation vendor, including the workflow, scalability requirements, delivery schedule, and quality standards. These should be clear, measurable, and agreed upon by both parties before any work begins.
Evaluate Several Companies: Make a list of data annotation vendors. Rather than looking just at their websites, evaluate their company histories. Look for press releases, media coverage, past clients, etc., to get an idea of how established they are and the scale they work at.
Evaluate Each Vendor’s Tools and Systems: Each vendor will most likely have its own in-house tools and systems. Ask to see examples of them in action. Do the systems make the job easy? Do they look like they should work well for large amounts of data or are they likely to be error-prone? If the vendor is crowdsourcing work, are the systems secure and does the company collect confidentiality agreements/NDAs for all of their annotators? Do you feel confident that your company’s data will be protected? Ask each vendor what their quality control system is. How do they guarantee that the data they are processing?
Start Small: Ask the vendors on your shortlist to commit to a small ‘proof of concept’ project. This will allow you to see some sample data, and also find out whether they are capable of delivering on time. If the sample project goes well, you can start to scale up. For some annotations, such as sentiment data, there is an element of subjectivity, and that’s acceptable. A face that seems “very happy” to one person may be judged as just “happy” by another. That’s why having a high volume of annotations helps, since a large number of ratings helps to smooth out differences of opinion. In scientific and medical models, there is far less room for differences of opinion.
If you are considering outsourcing data annotation, talk to the team at Shaip about your project. The annotation experts have experience with many AI projects, including deep learning, chatbots, and predictive models, and can help you get your project off to the best possible start.
Author Bio:
Vatsal Ghiya is a serial entrepreneur with more than 20 years of experience in healthcare AI software and services. He is a CEO and co-founder of Shaip, which enables the on-demand scaling of our platform, processes, and people for companies with the most demanding machine learning and artificial intelligence initiatives.
Wherever you are as you read this blog, I want you to take a moment and think back to how you went about your day, every small decision you consciously and unconsciously made, to get to where you are right now.
Perhaps, just like every student ever, you got caught up in assignments till late last night and could barely make it to the last bus leaving for your college that’d just about get you to class in time were it not for your mother who screamed, “IT’S ALREADY 8:00 A.M YOUR CLASS STARTS IN 20 MINUTES! “, jolting you back to reality.
Or maybe you’re a working professional and your boss declared, “It’s a slow day at work today fellas” so you’re just looking to brush up on your Deep Learning skills on the side.
Or maybe you’re a hobbyist, just here for the fun of it.
Big Data Jobs
Whatever the case may be, and however you got to read this blog today, I want you to think if you would still be here reading and understanding any of this if:
i. You could see everything but without the ability to interpret what you see. ii. You could hear everything around you but without the ability to understand any language at all.
That seems quite harsh. But clearly, the answer is no.
You wouldn’t get out of bed if your brain hadn’t interpreted whatever your mother yelled about it being 8:00 a.m and your class starting in 20, because you obviously wouldn’t know what any of that means or why that is a problem. For you, it would just be incongruous noise.
You wouldn’t have made the decision to browse Deep Learning concepts if it weren’t for you understanding what, “a slow day at work” means.
In, the same manner you wouldn’t be able to read or process any word of this blog because your brain wouldn’t know what a ‘word’ is in the first place.
All your brain would see is white, with some black interspersed.
You see, interpreting what you see, and what you hear, is something that we take for granted but cannot possibly survive without.
In the Deep Learning world, we have a fancy term for this. We call it, Natural Language Processing.
To learn more about Deep Learning, you can try out the “Practical Deep Learning with Keras and Python” online tutorial. The course comes with 3.5 hours of video that covers 8 vital sections. These include theory, installation, case studies, CNN, Graph-based models and so much more! This course is especially a helpful tool mainly if you are a beginner.
Also, you can opt for a similar online tutorial “The Deep Learning Masterclass: Classify Images with Keras”. Like the above, this course requires no prior experience either. The tutorial brings you videos of 6 hours where it covers 7 impressive topics that are important in understanding the concept deeper.
Natural Language Processing includes text processing (like the words in this blog), audio and speech processing.
Before we discuss how we can enable a machine to interpret speech or process text, we need to understand some concepts. I recommend you go through our previous blogs in this series for a seamless read. Right.
Consider a sentence from this blog itself.
“ In the Deep Learning world, we have a fancy term for this. We call it,Natural Language Processing.
Natural Language Processingincludes text processing (like the words in this blog), audio, and speech processing.”
Now consider this modified sentence and note how it will still make absolutely perfect sense to you,
“In the Deep Learning world, we have a fancy term for this. We call it,Natural Language Processing.
Itincludes text processing (like the words in this blog), audio and speech processing.”
The difference, of course, is just that I replaced Natural Language Processing with an ‘It’.
What do we learn from this? We learn that we interpret subsequent words based on our understanding of previous words.
We could immediately understand that ‘It’ refers to Natural Language Processing.
We don’t read one word, throw everything away, and then read another word and start thinking from scratch again.
Our thoughts have persistence.
For example, if you are developing an application that requires you to automatically calculate player runs in a game of cricket from the live telecast, you would first need your application to judge how many runs were scored (whether it was a 4 or 6 or a single) and then you would need to have the context from previous frames that would tell you WHICH PLAYER scored those runs so that you can add that many runs to the total for THAT PLAYER. It is difficult to imagine a conventional Deep Neural Network or even a Convolutional Neural Network could do this.
This brings us to the concept of Recurrent Neural Networks.
In the above diagram, a unit of Recurrent Neural Network, A, which consists of a single layer activation as shown below looks at some input Xt and outputs a value Ht.
A loop allows information to be passed from one step of the network to the next. When we unroll’ this loop architecture, we can see that basically, each unit passes a message to a successor, like a sequence.
Past inputs, thus influence decisions made on the present outputs. This is the key that was missing.
When the network begins to see things as ‘sequences’, we can immediately see that this solves our network’s initial problem of retaining or understanding ‘context’.
A question then, which should crop up in your mind is, from how far back in the sequence can a Recurrent Neural Network hold a context?
To reframe this, the question in focus is from how far back can past inputs, Xt, still influence the outcome of the present input activations Ht?
Well, unfortunately, not so far back. But let’s understand everything with example because we believe examples are the best way to learn.
Consider, we are trying to predict the last word in “when birds flap their wings they …”, we don’t need any further context, it’s pretty obvious that the next word is going to be fly. In such cases, where the gap between the relevant information and the place that it’s needed is small, Recurrent Neural Networks can learn to use past information.
But if you consider a sentence as complex as the one we took as our example at the very beginning,
“In the Deep Learning world, we have a fancy term for this. We call it,Natural Language Processing. Itincludes text processing (like the words in this blog), audio and speech processing.”For this, we have to go even further back.
Now if we were to predict the word after ‘ and ‘ in the sentence above, we would first need to know what ‘It’ refers to.
Unfortunately, as that gap grows, Recurrent Neural Networks become unable to learn to connect the information.
It’s entirely possible for the gap between the relevant information and the point where it is needed to become very large as in our example.
This is why we need LSTMs. Because LSTM units are perfectly capable of ‘remembering’ long term dependencies in a given sequence. In fact, LSTM stands for Long Short Term Memory.
Of course, to achieve this complex behaviour of being able to ‘remember’ context in ‘memory’, an LSTM unit also looks quite overwhelming in comparison to our Recurrent Neural Network.
The mathematical details are beyond the scope of this tutorial. A more detailed explanation of LSTMs will be covered in the coming blogs.
For now, let us move on to the final and the most interesting part of this blog, the implementation.
Implementation of LSTM with Keras
For this tutorial blog, we will be solving a gender classification problem by feeding our implemented LSTM network with sequences of features extracted from male and female voices and training the network to predict whether a previously unheard voice by the network is male or female.
Step 1: Acquire the Data
The data for this tutorial can be downloaded from here.
Step 2: About the Data
After downloading the data, which should be about 1.4 GB, you will notice 4 folders inside labelled:
Male: Contains 3971 male audio files from different speakers (Male Training Set)
Female: Contains 3971 female audio files from different speakers (Female Training Set)
Male Test: Contains 113 male test audio from different speakers (Male Test Set)
Female Test: Contains 113 female test audio from different speakers (Female Test Set)
The average duration of all audio files is about 5.3 seconds.
Step 3: Extracting Features from Audio (Optional)
Audio files are also one long sequence of numbers. But this sequence tends to be extremely long, almost 40,000 sequence numbers long, and because of our limited RAMs, we need to find more efficient ways to represent this sequence (audio). This is done by extracting the pitch of the audio along with something called the Mel Frequency Cepstral Coefficients (MFCCs) of the audio which is just a mathematical transformation to give the audio a more compact representation. Because we are dealing with audio here, we will need some extra libraries from our usual imports:
import numpy as np import librosa import os import pandas as pd import scipy.io.wavfile as wav
In case this throws you an import error, run the following line in your command prompt:
pip install librosa pip install scipy
Next, specify the paths of the Male and Female folders ON YOUR MACHINE in the variables accordingly:
path_male = "C:\\Users\\Bhargav Desai\\Documents\\GitHub\\gender-classifier-using-voice\\Data Preprocessing\\Male\\"; path_female = "C:\\Users\\Bhargav Desai\\Documents\\GitHub\\gender-classifier-using-voice\\Data Preprocessing\\Female\\"; mfcc_col=['mfcc'+ str(i) for i in list(range(110))]; def main(path,gender): df = pd.DataFrame() print('Extracting features for '+gender) directory=os.listdir(path) for wav_file in directory: write_features=[] y, sr = librosa.load(path+wav_file) fs,x = wav.read(path+wav_file) print(wav_file) mfcc_features=get_mfcc(y,sr) write_features=[pitch]+mfcc_features.tolist()[0]+[gender] df = df.append([write_features]) df.columns = col df.to_csv(gender+'_features.csv') def get_mfcc(y,sr): y = librosa.resample(y, sr, 8000); y = y[0:40000]; y = np.concatenate((y, [0]* (40000 - y.shape[0])), axis=0); mfcc=librosa.feature.mfcc(y=y, sr=sr, n_mfcc=10,hop_length=4000); mfcc_feature=np.reshape(mfcc, (1,110)) return mfcc_feature main(path_male,'Male') main(path_female,'Female')
Although this part seems complicated, do not worry. It is okay if you do not understand this part of the code.
All you need to remember is that this part of the code gives you a compact representation of the audio sequence for both the genders.
It is completely okay to skip this part for now.
Consequently, we have successfully reduced our audio sequence length from 40,000 to 113!
The outcome of this should be two .csv files named male_features.csv and female_features.csv in the same directory containing a compact representation of our audio sequences where each sequence is of length 113!
In case you encounter compatibility errors, please comment on your errors down below or directly download the feature extracted .csv files by clicking here.
Step 4: Labelling & Preparing the Data for LSTM Model
The outcome of the previous step was that we got compact representations of male and female audio sequences which might give you the impression that the data is ready.
However, we are missing one critical element: Labels.
Before we can train, we need to label our data so that our network can learn the difference between males and females from our data.
We do this by first loading our data from the two .csv files we have obtained:
The pd.concat() function joins two DataFrames, X.sample(frac=1) randomly shuffles the rows of the joined DataFrame and the XTrain.values function converts the DataFrame to a Numpy array X_Train.
Note: Here X_Train is our training set and Y_Train are the labels that we separated using the XTrain[‘label’] function.
Step 5: Building an LSTM Keras Model
We have come to the conclusion of this blog where we will show you how to create an LSTM model to train on the dataset we have created.
We are now familiar with the Keras imports and Keras syntax. But we’ll quickly go over those:
The imports:
from keras.models import Model from keras.models import Sequential, load_model from keras.layers.core import Dense, Activation, LSTM from keras.utils import np_utils
As soon as you run this code, you will get an error. The error will read:
ValueError: Error when checking model input: expected lstm_1_input to have 3 dimensions, but got array with shape (7942, 112)
This happens because of the LSTM implementation in Keras expects sequences from you as input. A sequence has an additional dimension of ‘time’ in addition to the number of samples and features that are called ‘timesteps’ in Keras.
To rectify the error, just add this line of code before you start building the model:
Here, our number of ‘timesteps’ which is basically how long our sequence is in time is equal to 112 with the number of samples 7942.The result of running the code after the modification will show you a screen like this:
This indicates that our LSTM network is indeed learning how to differentiate between a male voice and a female voice!It can be seen that the accuracy is increasing consistently (~81% in just 20 epochs!) and the loss is also going down.
X_Train = np.reshape(X_Train,(7942, 112, 1))
On a concluding note from our side, we’d like to ask our inquisitive readers to see how much the accuracy goes up by when you train it for more epochs. At Eduonix, we push our students to go that extra mile so we’d also like to encourage all our readers here to see if they can use the code used in this tutorial to record their own audio and see if they can get our model to predict your gender correctly!
Many aspiring Data Scientists, especially when self-learning, fail to learn the necessary math foundations. These recommendations for learning approaches along with references to valuable resources can help you overcome a personal sense of not being “the math type” or belief that you “always failed in math.”
More Resources for Women in AI, Data Science, and Machine Learning; Speeding up Scikit-Learn Model Training; Dask and Pandas: No Such Thing as Too Much Data; 9 Skills You Need to Become a Data Engineer; 8 Women in AI Who Are Striving to Humanize the World
The demand for analytics skills and talent has never been higher. As the workforce continues to evolve, so do the technology and skillsets needed. Learn how the Millennium Bank partnered with SAS to customize a development and training program that improved skills, knowledge, and retention.
Out of all the NoSQL database types, document-stores are considered the most sophisticated ones. They store data in a JSON format which as opposed to a classic rows and columns structure.
Flooding and unbearable heat have made many regions uninhabitable. Hurricanes and tropical storms often ravage the landscape, combining with particulate pollution to make the air outside almost unbreathable. Overpopulation, along with lack of habitable land, has forced many people to live together in cramped spaces, creating horrid living situations.
And, worst of all, there is a lack of food.
It is impossible for humans to create enough food to satisfy the needs of a population of over 9.4 billion people. Farms just aren’t efficient enough. Even in 2021, there were billions around the world who suffered from hunger. But with an extra 1.7 billion mouths to feed, farmers are presented with the impossible task of quickly increasing their crop yield to accommodate for extra individuals.
2070 is a nightmare. Climate change, overpopulation, and worst of all, no food — our world is a horrid place to live. If we had taken action earlier, 2070 might not have been so bleak and devastating.
Let’s rewind back to 2021. We need to find ways to counteract this disastrous future. One key way to fight this possible future is through optimizing our current farming methods.
Farming has come a long way, with new equipment and technology allowing for higher-quality crops and much higher crop yields. There’s no doubt that today’s technology is loads better than farming technology a hundred years ago.
But we can use technology to make today’s farms even better. To do so, let’s first look at how current farms are underperforming.
How Humans Prevent Farming from Working Efficiently
Farms today are growing to rely heavily on technology; high-end farms use robots, sensors, and drones to monitor crops and provide necessary nutrients, ensuring that all crops are as healthy as possible.
But even with all of these resources, much of current farming techniques are inefficient. Farmers have to manually diagnose when to harvest crops by looking at plant moisture and often have to estimate how much of necessary resources, like fertilizers, is needed.
Essentially, much of the farming process requires human intuition. And if there’s one thing that we know about humans, it’s that we tend to make a lot of mistakes.
Trusting human intuition, for the most part, has allowed current farms to experience a lot of success. But there’s always downsides. Going by human decisions for problems like fertilizer estimation might lead to shortages in necessary fertilizers, leading to a poor harvest.
This problem leads to impacts on crop yield, and with a steadily-increasing population, it’s important that farms do not waste any crops whatsoever. Every crop should be put to use to feed as many people as possible.
Farmers need to find a way to use technology to help ensure that farmers buy exactly the amount of fertilizers needed and crops are not wasted. In fact, there’s a whole field devoted to the use of technology towards solving such problems called precision agriculture.
Precision Agriculture + AI = Superpowered Farms!
Drones are a huge part of precision agriculture, giving aerial diagnosis of crop health while efficiently watering and fertilizing plants! Source
Precision agriculture revolves around the idea of making farming as efficient and precise as possible using sensors, robots, and other advanced technologies. Recently, precision agriculture has been employing the capabilities of artificial intelligence (AI) to achieve its goal of optimizing farming methods.
Artificial intelligence deals with using computers to predict or classify information given some data. Simplifying this a little, think of an AI as a person that reads a book on a subject and uses what it learns to perform a task. An AI that can classify between dogs and cats, for example, read a ‘book’ on the differences between dogs and cats, enabling it to recognize what makes them different.
The same idea can be applied towards agriculture. If an AI ‘reads a book’ on a specific topic in agriculture, it can perform the tasks that it read about at superhuman speeds and accuracies, making agriculture techniques both more convenient and more effective.
The proper technical process involves using an AI algorithm to analyze data from farms to optimize a specific task. The algorithm scans through the dataset, using information about the farm to perform necessary tasks.
Current sensor capabilities coupled with weather forecasts give us a plethora of data to create a model, from soil nutrient information to humidity levels. Using all of this information, we built an AI to optimize farm fertilizer purchases.
Using Neural Networks to Optimize Fertilizer Purchases
Fertilizer is an essential need for farms, but there’s some problems with purchasing it. If too little fertilizer is purchased, then a portion of the crops won’t get necessary nutrients, leading to a huge decrease in crop quality and yield. If too much is purchased, then farms lose money that could be devoted towards solving other problems.
We want to figure out exactly how much fertilizer a farm needs to get the best crop yield possible. This saves potential declines in crop quality while preventing overspending.
Luckily, AI is great at predicting things! We decided to use a neural network to solve this problem and predict necessary quantities of fertilizer necessary to optimize farming.
But, uhh… what is a Neural Network?
A neural network is a type of AI algorithm that’s really good at prediction and classification. Often, they’re visualized through a graphic of interconnected circles called nodes.
The nodes on the left-hand side are collectively called the input layer. Here, information is inputted into the neural network. In our problem, this information would be quantities like soil data, temperature, and other farming-related data.
The middle section, or the hidden layer, allows a neural network to develop a more in-depth understanding of the information. The larger a hidden layer (or the more nodes it has), the more complex information it can learn.
Finally, the output layer on the rightmost side returns the (you guessed it) outputs. Our problem involves outputting a certain number of pounds of fertilizer necessary.
Each of these nodes are stored in a computer as numbers called parameters. The end goal of the neural network is to learn parameters that are really good at predicting quantities of fertilizer necessary for a farm. It does this by scanning large amounts of data.
Data goes through the input, hidden, and output layers (in that order) to create outputs. The model then looks at its outputs and sees how incorrect its predictions were, changing its parameters to generate more accurate predictions.
Now, how do we use this for farms?
We created a model dataset containing features like average monthly temperature, wind speed, rainfall, humidity levels, and soil nutrient composition. The dataset also contained information about the optimal pounds of various types of fertilizers used given various weather and soil nutrient conditions.
Our example dataset of what crop data might look like. These values are not meant to emulate real examples and are completely random.
A quick note — this dataset used completely random values. None of the data is accurate; it was used to give an idea of how real data can be substituted into the dataset and used to generate meaningful predictions.
Using this random dataset, we trained a neural network with 9 input features (corresponding to a farm’s different types of environment data), 32 nodes in the first hidden layer, 16 hidden nodes in the second hidden layer, 8 nodes in the third hidden layer, and 3 output nodes (corresponding to three different fertilizers).
Our code for the neural network that we used, programmed in PyTorch.
Because of the randomness of our data (and our 24-hour time constraint), the model was not able to learn any meaningful information and had rather high loss values. Regardless, the model still successfully trained, showing that, with real data, a model that can predict pounds of fertilizer needed by a farm is easily trainable.
With more development time and clean, meaningful data, this model can become a powerful tool for farmers, ensuring that crops always give the highest possible yield and that no extra money is spent on acquiring necessities like fertilizers.
Advantages of Our Solution
Creating such AI has advantages not only for large farms, but for small ones as well. Here’s some of the key benefits:
Easily Modifiable. The neural network does not necessarily have to be used to predict pounds of fertilizer that a farm needs. With a few tweaks, the model can just as easily predict crop yield given weather conditions. It can also easily accommodate for more features about a farm or its climate.
Scalable. The model scales really well to large datasets collected by large farms while also working well with smaller datasets.
Ready Implementable. This neural network, after some tuning, can be shipped out to farms for usage within months, or even weeks. After training on some better data, the model is good to go!
Feasible for Smaller Farms. The use of non-complex AI algorithms makes this a useful product for smaller farms that cannot devote cost towards managing large AI systems.
Through precision farming and AI, 2070 does not have to be a desolate landscape of horrors; with optimized farms, humans will be able to feed and sustain themselves for centuries. Whether it be aerial crop surveillance or weed detection, these AI-powered optimized farms are slowly becoming realities. And, with our neural network’s capability to predict necessary quantities of fertilizer for farms, we hope to help contribute to the future’s smart farms, making our 2070 a happier place!
This article was written as part of a hackathon atThe Knowledge Society, where my team (Aashvi,Azam,Lillyand I) was challenged to create a pitch for how AI can be used to solve a tangible, real-world problem in under 24 hours. Check outour project Github herefor the full solution.
Thank you for reading to the end! Don’t forget to leave a ? as well! Connect with me onLinkedInif you want to talk about AI or any emerging technologies.
365 Data Science 
