… and your data warehouse / data lake / data lakehouse. A few months ago, I talked about how nearly all of our analytics architectures are stuck in the 1990s. Maybe an executive at your company read that article, and now you have a mandate to “modernize analytics.” Let’s say that they even understand that just…
I have been in data (in the fuzziest sense of the word) since about 2009, whether that means data engineering, management, analysis, strategy, or visualization. Over that time, things have changed drastically. In my first “real” data position, I was asked to identify and organize fallout from a claim auto adjudication engine to identify ways to…
In the era of big data and bigger AI, businesses are relying more on the importance of real-time data processing and analytics. Streaming data is a powerful paradigm for handling continuous, unbounded streams of data in real time. However, despite benefits like reduced latency, improved responsiveness, and the ability to make data-driven decisions on the…
Welcome to the latest edition of Mind the Gap, a monthly column exploring practical approaches for improving data understanding and data utilization (and whatever else seems interesting enough to share). Last month, we explored the data chasm. This month, we’ll look at analytics architecture. From day one, data warehouses and their offspring – data marts, operational…
Most enterprises today store and process vast amounts of data from various sources within a centralized repository known as a data warehouse or data lake, where they can analyze it with advanced analytics tools to generate critical business insights. Modern data warehouse platforms such as Snowflake, AWS Redshift, Azure Synapse Analytics, and IBM Db2 are built with…
In the healthcare industry, surgical capacity management is one of the biggest issues organizations face. Hospitals and surgery centers must be efficient in handling their resources. The margins are too small for waste, and there are too many patients in need of care. Data, particularly real-time data, is an essential asset. But it is only…
While early science fiction shows like “Buck Rogers” (1939) and “The Fly” (1950) depicted teleportation technology, it was Star Trek’s transporter room that made real-time living matter transfer a classical sci-fi trope. While we haven’t built technology that enables real-time matter transfer yet, modern science is pursuing concepts like superposition and quantum teleportation to facilitate information transfer across any distance…
If you’ve never heard of dark data, you’re not alone. Setting aside the ominous name, dark data isn’t something that is inherently bad – although, in practice, it usually does end up this way. Dark data is usually unstructured data, though it can also be semi-structured or structured data that a business collects and stores but…
Data analytics and integration are the key components of building a data strategy. For organizations to have an effective data strategy, it requires the definition of measurable metrics and proper consideration of all data sources. An effective data strategy also needs to define how data can be moved from various sources to a location where…
Dark data remains one of the greatest untapped resources in business. This is due to the vast amounts of usable data that exists within an organization, but is not utilized or analyzed to serve a specific purpose. These untapped sources could include customer information, transaction records, and more. Since dark data represents missed opportunities for…
