Data Systems Artisan

In the rapidly evolving world of Large Language Models (LLMs), the ability to run these powerful tools locally on your hardware is likely appealing. Running AI locally grants you unparalleled control, endless customization, and complete data privacy. This guide will walk you through building your personal AI playground, assembling an... [Read More]

The concept of a data lakehouse is gaining serious traction, and for good reason! It combines the flexibility and cost-effectiveness of a data lake with the data management features and ACID transactions of a data warehouse. Let’s dive into building a practical lakehouse setup using some fantastic open-source and cloud-native... [Read More]

Popular data processing platforms offer users the ability to inject an external process into the data processing pipeline. The data flowing through the data pipeline is fed as input to the external process, while the output produced by the process is fed back into the pipeline. The external process runs... [Read More]

In an earlier post, we looked at how to create a Docker image for SciDB. The image built in that post followed the SciDB Community Edition Installation Guide very closely. The image is functional and a good learning resource, but not very efficient. The image uses around 6GB of space... [Read More]

One of the strengths of SciDB over other database management systems is its extensibility.1 SciDB allows the user to add new data types, functions, and operators. In this multi-part post, we discuss various aspects of extending SciDB. In this post we look at the available documentation and how to setup... [Read More]