Data Systems Artisan

The Power of Loading Data - Part 2

Multiple Files with Non-Integer Metadata

Posted in June 2016 by Rares Vernica

In part 1 of this multi-part post, we gave a short overview to the accelerated_io_tools plugin provided by Paradigm4 and showed how to automate loading multiple files by capturing both the data in the file as well as the data in the file name. In this post, we go a... [Read More]

The Power of Loading Data - Part 1

Multiple Files with Metadata

Posted in May 2016 by Rares Vernica

Loading data into a database is a very important operation. SciDB is no exception. Build-in SciDB is the vanilla load operator documented here. On top of that Paradigm4 provides an advanced loading operator as part of their accelerated_io_tools plugin. Similar to SciDB, this plugin is open-source but different from SciDB,... [Read More]

Getting Started with SciDB

Using SciDB Amazon Machine Image

Posted in April 2016 by Rares Vernica

The easiest way to get started with SciDB is using Amazon Web Services (AWS). There are multiple SciDB Amazon Machine Images (AMI) provided by Paradigm4 (the company behind SciDB). For each AMI, the type of Amazon Elastic Cloud Compute (EC2) instance recommended by Paradigm4 is pretty beefy. We recommend following... [Read More]

A Brief History of SciDB

From Academic Workshop to Commercial Product

Posted in March 2016 by Rares Vernica

SciDB was born out of the Extremely Large Databases Workshop (XLDB). The workshop was intended to bring together people from industry, the scientific community, and academia and address issues pertaining to extremely large databases. The SciDB name was first mentioned 2008 in the final report of the 2nd XLDB Workshop... [Read More]