Articles, Blog

Bridging Data Gaps Across the Global Agricultural Sector

good afternoon everyone I’m Sara Menker
I’m the founder and CEO of grow intelligence which is an AI company
focused on global agricultural markets we’re based in New York and in Nairobi
Kenya and I’m gonna talk to you about how AI is being used to transform global
agricultural and food markets as we as we know it so headlines in January
through May of 2019 there was a government shutdown Kenya feasts the
worst drought in 38 years there was over a million acres of cropland ravaged by
floods and actually as of today it’s closer to 5 million acres ebola pig
virus it’s called African swine fever destroyed the the swine population in
China about a third of it is gone and there was a revolution in Sudan now in
all of these headlines you actually see agriculture and food represented that
there is a headline that actually doesn’t mention it which is in Sudan the
Revolution was actually entirely driven by food prices this chart that you see
over here is a spike in sorghum prices in Sudan which is the key grain that
everybody essentially consumes and here you had a devaluation in the currency
this is the revolution the dates of the revolution are entirely tied to the
price of food this happens over and over and over again and we have not learned
to better understand and plan for our food and agricultural markets after
calculating the estimates from the various orange producing states we have
concluded the cold winter is apparently not affected so this scene is from the
cult-classic trading places with Eddie Murphy this specific scene is when the
US government the USDA the Department of Agriculture comes out and gives its
report of the orange and it does so once a month and markets
go completely crazy now this is from 35 years ago it still happens this way
today nothing has changed a process is manual it’s impossible to
find the information you need and so the markets once a month sit around and wait
for a report to be issued by one government agency that essentially
actually reports for the rest of the world because nobody else even has this
process in place and it essentially leads to a lot of information asymmetry
and inefficiencies in our global food systems this is essentially how people
used to not find information on agriculture and I I learned this
firsthand when I used to trade energy which is what my job was before I
started this company and then I tried to move into trading agriculture and I
realized how much more complicated it was than oil or gas or power or all
these other commodities that we we trade and so what we decided to do was build
an AI that’s built on three key principles domain expertise so for
agriculture you really need to know your stuff
meaning it’s impossible to build an AI to solve such a complex real-world
system without deep domain expertise so we built a team that has expertise in
hydrology plant science markets you know geography you name it that lets us build
depth and depth is necessary again agriculture is a complex real
world system we think corn and wheat and soybeans and the few things that we hear
about but reality is there’s tens of thousands of little commodities there’s
vanilla and black pepper there’s it’s the system is super complex so you need
depth in it because there’s a lot of relationships between them and the third
is diversity if you want to build an AI and essentially solves a complex
real-world system you need to actually build it on diversity so that it
represents the world that you are then trying to represent and so we
essentially built the company on these three principles and our product on
these three principles and it’s really reaped a lot of rewards
for us we started to see the results and the successes that I think are truly
driven by these three things now why do we do it I mean when the US government
shut down we actually replaced the USDA we as a company of 65 people at the time
in a span of one week generated 1800 forecast models that were actually more
than the number of models that the US government releases automated the
process released it for free to the public on the On January 11 there was no
report from the US government so we were able to step in at noon and release
these numbers through outlets like Bloomberg and Reuters and essentially
the markets continued but the markets continued because we were able to do
that now we didn’t do it because you know replacing the US government is our
goal but it’s one thing to remember that the u.s. is a very dominant player in
global agriculture and so what comes out of there really has ramifications
everywhere else in the world this chart that you see here is basically the
amount of product of corn rice soybeans and wheat just use that as a benchmark
globally as you can see in terms of volume the US and China are actually
quite similar but why the US is dominant is because when you look at it on a per
capita basis your Purple Line is North America and Asia is your Green Line so
yes while on a total production basis the amounts are equivalent on a per
capita basis they’re significantly lower so what happens in the US markets the
u.s. is a dominant player and so therefore has an impact on every other
market and these links are really complex to model and having one gap like
that can have all sorts of you know unintended consequences that feed
through an entire system so the world is much bigger and the reality is the rest
of the world doesn’t have what I even described which is the USDA process so
yes that is a little bit still needs you know some improvements I’m trying
nc etc but at least it exists there are many parts of the world that we’re
generating forecast for today India China Russia Kenya South Africa where
you don’t have that as an alternative and to have automated models that are
giving you signals on a continuous basis is really important to solve food
security challenges the world is much bigger than the US right and this is
something that we forget and this is really important especially in the
context of what’s going on today if I you know if you remember I just
mentioned to you that the floods that had a million acres of cropland ravaged
about a month ago that’s now five million acres that is the world’s
largest exporter of corn and soybeans what this means to global markets when
you are the dominant supplier and the rest of the world population is
distributed like this is that this will have a ripple effect literally for
probably two years and we’re just going to start to feel it what happens in the
next decade the lines that are already high keep getting higher you’re
flattening out in population so this is looking at population across the world
and so you have regions like Asia and Africa that are growing the fastest
they’re the youngest but also if you go back to this chart they’re also the two
regions that are producing the least on a per capita basis so how do we start to
equalize production around the world especially when we now live in a world
where the world trade order is in complete chaos do you continue to depend
on those same markets that you have brought you know food in front would you
bet your entire population on that things have to change but for things to
change we also can’t rely on doing them the way we have done them before we have
to make them significantly cheaper we have to make them more efficient we have
to make them more understanding and we have to make them really transparent and
so what we did is a company is we realize that the world of Agriculture
one thing to keep in mind is that it’s very public it’s actually because food
is politics most of the data around agriculture sits
in the public domain you can find it but good luck using it good luck
translating it it’s mostly in formats that are almost impossible to use their
PDF files with scanned images and then somebody throws handwriting and then
moves the tables around that’s the way that most of the data is reported but
when you bring it all together it gives you something super powerful so we
essentially harvest all of this data and we make it go through an automated
translation and transformation there and this is where you’re actually doing
language translation per and this is very domain-specific language
translation so it’s actually really hard to use off-the-shelf tools for this once
that’s happened kind of the key to what we do is our ontology we’ve basically
designed an ontology that where we can essentially index and classify this data
to make it searchable discoverable way easier to use it’s normalized and once
it’s normalized it starts to give you insights like you’ve never had before
we’re starting to understand a food system an agricultural system that’s
been around since the Neolithic Revolution for the first time today and
it’s kind of amazing because we’re just getting started and you know so we’ve
built as I said all sorts of predictive models we’ve taken the very
unconventional approach in that we actually make all our methodology public
which has confused people but the reason we did that is because we want to build
trust you cannot build trust in something that is so mission-critical to
the world without being open about how you’re doing it you can’t just say trust
me I got this because no one’s done this before so you really have to take a
humble approach to this process in some ways and so we have an API that powers
all sorts of products that product can be visual you can interact with it you
know through voice or you can even have chat BOTS
so depending on the complexity of what you want to do and who you are is how do
we help everybody better understand whose food systems organizations that
need it but also individuals so today we’re processing north of 55 million
unique data series that come in multiple languages and formats think of this as
each data series is basically a term that’s defined in our dictionary
agriculture this is what I mean that it’s super complicated this No
was about eight million at the start of last year at the beginning of this year
it was about forty million and now it’s actually north of sixty million as of
today and this continues to grow because agriculture is not just food
it’s whether its climate it’s our soil it’s people its culture you know as soon
as you start to understand what drives it the data that you need to model it
gets wider and wider more and more complicated we’re processing north of
six hundred fifty trillion data points a day now and this is why we’re starting
to generate some amazing insights that we just never had before in
understanding and deconstructing our markets so this is an example of AI and
machine learning and grow which is you know whether we’re doing crop yield and
supply forecast models where we’re using a series of machine learning models to
forecast in season yield so how much you know on any given day in India today we
for example for every county in India we can tell you how much wheat is being
produced how do we do that the first thing first problem we actually have to
solve is classing a classification of crops they don’t tell you what fields
grow what crop in India and in India unlike the u.s. where a field is 50
thousand acres you have two acre plot and someone’s growing potatoes onions
wheat and corn all side-by-side so how do you start to classify those images to
determine what field is growing how much wheat from there you can then start to
do a supply model to do this for the whole country of India it’s like doing
it for a continent it’s so complicated we build drought models so how do you
better predict drought we actually have flood models that helped forecast a lot
of so the markets today are using are we’re the only company in the world
today that has a forecast model that can tell you how much acreage is being
destroyed due to a flood or two droughts and you can do this real-time you can do
this anywhere in the world the other thing that really helped us is the
automation of our mappings through neural networks so when you’re building
a dictionary that’s so complex you cannot have humans sitting there doing
that mapping in the beginning you need it you absolutely need it and this is
where the human intelligence part comes in handy we really are a company that
was built on just brute force human intelligence
to start and then it was then taking that and mixing it and bringing together
a team of technologists that had nothing to do with agriculture and getting them
to work together and this is what I mean by diversity it’s not just diversity of
background you know where people are from or what they look like its
diversity of thought so how do you get people who are philosophers to work with
engineers and computer scientists market research analysts and as I said and this
is really important because agriculture is an interplay between supply demand
trade and price at the end of the day every single person in the world wants
to know where food prices are going if you’re a government that’s what you care
about if you’re a company that’s what you care about if you’re a trader that’s
what you care about if you’re a consumer that’s what you care about but
ultimately that one signal that you get is an interplay between I mean thousands
and thousands of models that underlie all of this and so what we’re building
is this we call it the you know grows unified model of Agriculture but it’s
basically think of it as building every single one of these models for every
crop for every region in the world and so if you have a slight drop in pork
production in China the ramifications are everything from soybean prices in
the US – what Brazilian farmers plant – do you now eat more chicken if you eat
more chicken than the chickens now eat corn they don’t eat soybeans I mean it
just starts to get so how do you weave through that system and get an answer
that tells you do I produce more chicken are people gonna eat more pork or do I
eat more fish like it’s really complicated and so building this
interplay and understanding this interplay is really necessary if we are
going to tackle the challenges that we have you know when we say food security
we kind of say it very blunt like you know just like it’s another term but we
always forget the second part of that term security it really is security in
most countries food security is a national security matter and somehow
these systems haven’t been built for national security purposes you know let
alone anything else and so we really have to start thinking of it so we
cannot all food security until we solve for
this problem and you know these supply demand trade and prices are constantly
constantly disrupted you have climate change that’s getting in the way you
have demographic trends that shape you know what we eat and demand but then
that gets kind of mixed in with cultures and norms and softer things you know it
doesn’t mean that every population that’s growing fast is gonna eat more
meat because some populations don’t so what else are they gonna eat so how do
you start to understand that it’s driven by socio-economic trends then you have
policy and this year has been last two years have been quite an example of how
policy can be the biggest disruptor to markets as well and we always forget
that we assume that policy makes markets function policy can also break markets
and then you also have just not so rational markets and this is the thing
you’re fighting against so modeling this in real time is actually quite necessary
and you don’t start to solve and tackle issues and challenges and plan for the
future without this and the world that we believe is possible the world that we
work towards is increased productivity not just production meaning let’s not
keep destroying our land and our forests to just keep producing more and more
let’s learn to produce more with less and that’s entirely possible
it’s entirely possible we need affordable food for all we can’t have
Sudan happening over and over again and talking about it or not talking about it
which is even worse we need less waste again to optimize a system for waste you
need to understand what you’re optimizing for and then you need a
healthier planet both our earth meaning we need to take care and better care of
our soils and our water but also our people so if you want a
healthier planet and healthier people you really need a much more optimized
food system and this is just the beginning thank you you

Leave a Comment

Your email address will not be published. Required fields are marked *