Data Engineering in the Age of LLMs
vishwanath Leading large data engineering teams in the age of LLMs always leads to many existential questions around the future and state of data engineering considering its current fragmented tooling ecosystem.…
Advance your AdTech Measurement: Practical Tips
Running large data platforms using 1P, 2P and 3P data involves measuring your AdTech spend and a clear strategy to execute. With the gamut of vendor solutions, agencies, ad exchanges…
DQ Framework
Data quality is one of the biggest asks from consumers of the data in large scale implementations. As the scale of the data grows, so does complexity and the plethora…
Autoencoders for Data Anomalies
With more and more emphasis on data anomaly detection and the proliferation of build/buy options, I’ve been exploring auto encoders for a few projects. In a nutshell, Autoencoders are a…
Laplace and the law of small data
One of my recent favorite reads is The Computer Science of Human Decisions by Brian Christian/Tom Griffiths. In the age of “big” data, encountering uncertainty and little data is also…
Spark AI Summit 2020 Notes
Spark AI Summit just concluded this week and as always, plenty of great announcements. (Note: I was one of the speakers at the event but this post is more about…