Procella is a horizontally scalable, eventually consistent, distributed column
store leveraging lambda architecture to support both realtime and batch queries
[1]. Let’s define those terms one at a time:
Horizontally scalable means YouTube can spin up more machines and Procella
will distribute queries to the new machines automagically.
Procella doesn't support strong database isolation levels (the I in ACID). Queries
support read uncommitted isolation which can cause dirty reads. A dirty
read occurs when a transaction sees uncommitted changes from another
transaction.
The lambda architecture means there are two write paths. The first path, called the
real-time write path, aims for low latency and writes into an unoptimized
row-store that’s immediately available for queries. The second, called the
batch write path, ingests large quantities of data into an optimized columnar
format. The query path merges changes from both real-time and batch storage to
unify results.
Distributed means the data is sharded across multiple servers.
A column store refers to how the database physically stores data. See the
figure below for how we might store a table with three columns: event_id,
time, and path.
Figure: The difference in data layout between a row store and a column store.
After implicating Lastpass as a culprit in browser slowness, I ripped it out
and replaced it with the significantly snappier 1Password. Lasspass slows down
the browser in a litany of ways:
Chrome compiles the Lastpass JavaScript bundle on every page load. Crucially,
Lastpass injects the script so that Chrome blocks the first contentful
paint. Lastpass takes 70ms to compile on a relatively high-end MacBook pro.
This means Lastpass adds a 70ms delay to all page loads.
The Lastpass vault is hilariously slow. Opening up account properties in the
Lastpass vault pegs the CPU at 100% for 15 seconds.
On a more subjective note, interacting with the vault and form-filling with
Lastpass feels much more janky than 1Password.
The migration to 1Password was surprisingly straightforward. The 1Password
documentation breaks down into two easy steps with nine total subtasks. I
completely migrated to 1Password in less than 10 minutes.
Lastpass performance
Let’s examine Lastpass performance on a simple page, example.com, to evaluate
the performance impact of the extension.
Figure: The performance timeline of loading example.com with Lastpass. Notice
that the "Evaluate Script" for Lastpass blocks the first contentful paint (FCP).
Onloadwff.js is part of the Lastpass extension.
The Ubuntu message of the day (MOTD) is a chatty affair. A MOTD
sends information to all users on login—A recent login message greeted me with
42 lines of questionable value.
Welcome to Ubuntu 18.04.2 LTS (GNU/Linux 4.15.0-1021-aws x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Mon Apr 20 01:01:24 UTC 2020
System load: 28.97 Processes: 10
Usage of /: 28.9% of 48.41GB Users logged in: 0
Memory usage: 61% IP address for enp4s0: 10.0.101.001
Swap usage: 0%
* Kubernetes 1.18 GA is now available! See https://microk8s.io
for docs or install it with:
sudo snap install microk8s --channel=1.18 --classic
* Multipass 1.1 adds proxy support for developers behind enterprise
firewalls. Rapid prototyping for cloud operations just got easier.
https://multipass.run/
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
* Canonical Livepatch is available for installation.
- Reduce system reboots and improve kernel security. Activate at:
https://ubuntu.com/livepatch
99 packages can be updated.
1 update is a security update.
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
*** System restart required ***
Last login: Sun Apr 19 03:41:04 2020 from 192.168.0.1
Gorilla is an in-memory, time series database from Facebook optimized for
writes, reading data in a few milliseconds, and high availability. Facebook
open-sourced the code as Beringei, but the maintainers archived the repo. At
its core, Gorilla is a 26-hour write-through cache backed by durable storage in
HBase, a distributed key-value store. Gorilla’s contributions include a novel,
streaming timestamp compression scheme.
Figure: The architecture of a Gorilla cluster. A cluster runs in 2 regions.
Each region contains multiple instances. Each instance contains shards that
store time series data.
I care immensely for having a snappy terminal startup and rebel against demands
to blindly source scripts in my ~/.zshrc.
The most recent challenge was Node Version Manager. NVM commits the mortal sin
of automatically adding itself to the ~/.zshrc file in the following
incantation:
The NVM script requires 800ms to load, a debt paid every time I startup a shell.
Consulting internet canon, we calculate how much time it’s worth spending to
reclaim the 800ms of shell startup.
One itch I’ve wanted to scratch for a while is to create a web-server from
scratch without relying on libraries and without first
inventing the universe.
I’ve also wanted a chance to take Go for a spin. I’ll cover how to create a web
server in Go using Linux system calls.
I often need to query complex things with Bazel, an
open-source build system from Google that focuses on performance and correctness
by enforcing hermetic builds. For a more complete list of examples, see the
official
Bazel query how-to.
Find all tests marked as size = small that have a database dependency
Google tests have a specific
size (small, medium,
large) with strict time-outs. If a small test exceeds 60 seconds, the test
fails. For tests involving a database, the tests need to be marked as medium to
avoid flaky timeouts.
