As big data is turning from a buzzword to an actual thing, the way we do business is becoming more and more data driven. That can be a good thing because it can open up a whole new perspective on the workings of a business case. But it can also be annoying because the amount of data we have to ingest and interpret is skyrocketing as the trend continues.
The big challenges in dealing with the flood of complex information are condensing it to the parts that matter and helping people navigate to the bits and parts that really matter in their current situation. While a lot of tools somehow try to condense the information by sorting it or visualizing, we decided to see how we could actually mold it into a narrative that helps to grasp it. But prying insights out of a spreadsheet can become a needle in a haystack game, especially if the information you are looking for is multivariate. So the first impulse would be to visualize and condense the information. But a graph is always going to come with some biases - it needs to be scaled and cropped and coloured and all of this will influence the way we read it.
We have been trying to wrap our heads around this problem for a long time. Like how could we help people with narrow timeframes and hourly rates in the hundreds to save time while dealing with big data? We figured, that our automated narratives could help because they convey more information than naked or compressed data. For example, they can include conclusions or focus much more on a particular subset of the data, to direct the reader to the bits, where he or she will really have to take a closer look while filtering the ones that are just not so relevant at the moment.
So after pitching the idea to some of our most visionary customers, we came up with two approaches to harness Natural Language Generation in Business intelligence and Big Data Scenarios: The first was to facilitate ingesting BI data into our NLG platform.
We decided to explore what customers would need in this space, by offering some mathematical functions in an additional service. Because we are very cool developers, and our example data was about cryptocurrencies, we called it “Cryptwalk”, alike the thing, Snoop Dogg does.
Customers would be able to throw their data against a particular endpoint, and this module would calculate all sorts of statistical, arithmetical and market based measures and predictions and simply add them to the data. All the highs, lows, trends and meta observations of a trading day, there for them to pick.