Swimming in the Data Lake

By: Tony DePrato | Follow me on LinkedIn

Educational organizations are face with the constant influx of seemingly new technology. This creates pressure (even a demand) to review, to compare, to challenge, and often, to change.

The business around software is not understood nor recognized by most educational institutions. Schools trust. Schools hope. Industry, generally, tries to profit. The constant churning of technology companies through mergers and venture capital initiatives creates an unstable environment. This environment breeds havoc and forces decision makers to ask: Will your vendor be your vendor tomorrow?

The game is rigged. But. There is way to take the game back. There is way to design flexibility into the ecosystem and empower decision makers to relentlessly negotiate for the deals they need, and when they need them.

For years we have focused on controlling the box, the physical interface, and the platform. In most cases, these concepts are now irrelevant. Software as a Service (SAS) has evolved to allow a tablet to carry the power of a laptop, and a laptop the power of a GPU driven workstation. SAS is just the beginning, in fact, it is only a replacement for the platform. The heart of the system is the data, and the data controls the decisions more than anything else.

The next evolution for education, K-12 and above, is to adopt new standards allowing the organization to choose their modality but maintain the standard of communication.

Educational institutions need to build a data lake, or data repository, using data from all their vendors. Any vendor that cannot meet a few basic standards, needs to be eliminated from the pool of options. These standards would be simple, and would include:

• Data, all data, can be exported in a single data pump when required
• Data, all data, can be exported into at least one or all of the following formats: csv, tab, sql, or xlsx
• Downloaded Data, all downloaded data, will only have encryption if the client chooses
• APIs and other methods to sync real time data are optional; even if these tools exist, the data export requirements must be maintained

By insisting these standards be met by vendors, educational organizations will not only be able to constantly analyze all their data, they will be able to recreate themselves when they choose. Vendors will not hold the fear of data loss, or opportunity cost, over the decision makers.

The only remaining conundrum is: how do we show every school how to do this, and how to find opportunity within this new environment?

Source: IT Babble Blog and Podcast