QUERY / ACT AND CONNECTIVITY WITH DATA SCIENTISTS

You’ve got data, now what?

One of the biggest challenges organizations face is how to retrieve data, where to retrieve it from, how to store it in an efficient (in time and cost), homogenized, performance-optimized way, so it can be later consumed from several business units and use cases.

And when they think it’s all done, you find yourself with a “data warehouse/lake/mesh” that is not capable enough to handle the queries you want, or queries take too much time to run because you don’t have a distributed query engine, or they turn out to be too complex for your data model, or the way you stored doesn’t play nicely with ML pipelines or scientist notebooks.

If you chose Nucleoo, those topics are already taken care of with the Query and ACT modules.

Q&A mode…

From the foundations of Nucleoo, Query and ACT are the natural choices for data querying and data alerts.

Query is a module that provides different interfaces (from Python classes, to API calls or graphical interfaces) to query the data the way you want. With our customized approach, Query is designed to your specific needs, optimizing, together with the storage engine, the most used cases on your analyses, providing top performance, specially on your most common queries, even on huge amounts of rapidly changing data. Query on every field, grouping, filtering and running aggregations in a powerful, yet simple way.

For data scientists, it’s the “turbo” that their analysis needed. The infrastructure and storage are designed in a way horizontal scaling and data sharing between processing nodes provide a high level of throughput to get you the data as fast as possible, so you can start working on your analysis now, run A/B tests faster and testing different scenarios without having to wait long periods of time between tests.

And all this while using your favourite tools: Python ML pipelines, Scikit learn, Pandas, Spark and Jupyter notebooks.

… or ring me when it happens

But what if you don’t want only querying, but being alerted when something happens? ACT comes into play. ACT alerts you whenever a KPI, configured by you, goes under/above a threshold with a set of other conditions. It’s a way of putting a Query into “configure and forget” mode.

Be alerted when the number of items sold for that group goes below X for more than a week, when your satellite detects that your area of land has been dryer than your 5 years monthly average, or that the sensor in your front wing has been almost flat for more than 2 laps.

Query and ACT are powerful tools included in Nucleoo, that allow you to know what’s happening, when it’s happening, and provide you and your data scientist all the help you need to know why it’s happening.

Written by
Rubén Trujillo
CTO Bi4 Group