Session language – English Target audience – Developers, Data Scientists, R&D

I plan to discuss three archetypical war-stories about fitting in memory. In each of them, I'll describe both the technical challenge and the human biases that needed to be overcome to arrive at sound solutions.

One aspect of handling big data is that typically a problem's dataset does not naively fit into RAM. Three episodes I'd like to discuss:

• Choosing a data-at-rest format for large dataset without reinventing the wheel.
• Choosing the right in-memory format for a sparse shortest-path matrix, when the dense version would be prohibitively big,
• How to chew thousands of larger than 1GB JSON files without swallowing them whole.

Administrative notes:

I strongly encourage you to turn on the subtitles. I'll be available at the live chat during the session. We will switch from the video to the Q&A breakout room at 12:10 Feel free to follow up with me through uri AT immunai DOT com

I'll discuss the problems, their solutions and the mistakes I made along the way