14 thoughts on the economics of the open-source data space and how to become the next MongoDB or Databricks — The data space is booming, with companies like mongoDB (valued at 18 billion USD), databricks (30 billion), or Confluent, and many others. The startup space is overflowing with money and lots of founders want a share of the pie. But in my opinion, the data space is set up to…

“The advancement and diffusion of knowledge … is the only guardian of true liberty.” — James Madison Some people think data is all there is and that it will dominate the future of our world. Some think data isn’t that important after all, and that change will happen slowly. The…

The Scotts have a saying: many a little makes a mickle. Some jobs aren’t fun, but you must plod along to get them done. Becoming “data-driven”, and making data-inspired decisions as a manager, as a knowledge worker, and as a company definitely falls into this category. There is no sense…

Reading is a superpower, knowledge a person has accumulated over years you get to ingest in minutes. But it also comes with dangers, the danger of ingesting crap, to spend hours filtering through content that isn’t good at all. And yet, from time to time, you stumble over unique articles…

On the 7th of January 2021, I started my newsletter about data for one reason: Because of an open-source tool called dbt. I got introduced to dbt early, together with an introduction to the fishtownanalytics mentality around ELT. …

Why datasetcentric visualization is better than query-based or semantic layer-based visualization for companies that want to move fast. — Do you know the classic “semantic layer”? It’s a super confusing term. The classic semantic layer is called a “layer” because it sits between your data store and your visualization. The classic semantic layer usually was built into your visualization tool, your Pentaho, your Microstrategy, etc.

Chatting with a PDF file? No problem, build it on top of a LLM. Chatting with your database without SQL? Same story. Building a chatbot into your Slack channel? These types of applications utilize LLMs like ChatGPT to build their backbone. Yet precisely, this backbone work is heavy data lifting…

I don’t like to dwell on the past. And a lot of times, it is essential to make mistakes yourself. And yet, I find myself recommending many practices to new data engineers. Because I felt the pain personally, and it wasn’t fun. I spent two months early on my job…

A simple explanation of what vector databases & embeddings are and why they are so hot right now! — Gazillions of dollars are being poured into vector databases like Pinecone and Weaviate. But why? Both companies were founded already five years ago. To understand, you need to understand the essence of three things, vectors embeddings vector databases Luckily, I got a story ready for you that makes it easy…