Some thoughts on configuring web servers
If there’s one thing that has always made me annoyed running a web hosting and services business it was the low level details of configuring virtual hosts in Apache and every other web server on the planet.
It’s all scriptable, but it’s error prone and completely graceless.
Users want to be able to define their own rules.
Apache configuration syntax, when included, can break the entire configuration. It’s not dynamic. Reloads in a hot web server can be expensive.
Ngingx and Lighttpd are marginally more consistent, but still stink at delegating.
Configurations are sometimes order-dependent, sometimes evaluated root to leaf node, sometimes leaf node to root, and sometimes require recursing into the request handler to make decisions based on on “what if” scenarios.
I’d willingly trade a lot of power in configuring a web server for something simple and able to be delegated to users.
There are some basic requirements:
- Ability to configure redirects (and custom responses) for specific URLs and for entire subtrees of URL space. (I’m of the opinion that this should often not be handled at the application layer, since it’s most often needed to deal with changes to URL structure during application upgrades and transitions.)
- Ability to map URLs to handlers located within the document root, without exposing the filenames of those handlers. (Thank you, PHP, for moving us backward 5 years in URL structure in an effort to teach us how simple deployment should be.)
- The ability to direct entire subtrees to a handler.
- The ability to direct entire subtrees to a handler if the request is not satisfiable with a url-to-path mapping.
- The ability to direct requests to a handler if url-to-path mapping yields a file with a particular suffix (or perhaps indirectly via MIME type)
- The ability to tweak url-to-path mapping if url-to-path mapping yields a directory.
- The ability to add a variable passed on to a handler at any point in the subtrees of URL space, including setting it to a value from any part of the request headers, including a fragment of the URL.
And operationally, I want to be able to delegate the configuration of entire virtual hosts and preferably also subtrees of URL space to users, and have them only able to break the parts delegated to them.
1944 Red Velvet (cup) Cake
This is a red velvet cake made in a WW2 era way, using beets for moisture and color. The trick to getting good color rather than mud is to keep the batter acidic: lemon and buttermilk and a complete lack of alkaline leavening are what make this recipe unusual.
Boil two medium beets and puree. (You need one cup)
Cream two sticks of butter with a cup of sugar. Beat in two eggs as completely as you can.
Mix two tablespoons of lemon juice with 3/4 cup buttermilk. Add a cup of the beet puree.
In a bowl, mix a cup of flour and a quarter cup of natural (non-Dutch process) cocoa powder. (I used Hershey’s).
Beat the three mixtures together, adding some of the butter, egg and sugar mixture alternating with some of the beet and buttermilk mixture.
Pour into greased cupcake pans and bake at 350 until a toothpick or straw comes out clean.
This will be a soft, moist cake, almost custard. It released from the pan easily for me, though my cupcake pans are cast iron and a little unusual.
I used most of my batter as a layer under a cheesecake, but that’s a story for another time.
Perhaps it is broken, the cover of your diadem […], darkness collar […]?
An excellent and beautiful work by my friend @vruba.
Perhaps it is broken, the cover of your diadem […], darkness collar […]?.
Configuration injection in node.js
[gist id=3927254]
The world is a complicated place
Part 1 of an ∞ part series
:; cal 9 1752
September 1752
Su Mo Tu We Th Fr Sa
1 2 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
using jsrender templates with jQuery UI
Load the script:
[gist id=1343709 file=head.html]
Define your template:
[gist id=1343709 file=template.html]
Then connect it with a jquery UI dialog:
[gist id=1343709 file=action.js]
Cephalapod Surprise Chowder
Chop four slices of bacon and start cooking them over medium heat.
Chop one small onion, three small carrots, two sticks of celery.
Add them to the cooking bacon, with the fat.
Let them cook until the onions start to go transparent.
Add a cup or two of beer.
Add 3 or 4 fingerling potatoes, cut into small bite sized pieces.
Add water to cover and let this simmer until the potatoes are soft.
Chop four or five small squid into half-inch square pieces. Tentacles can be left in larger pieces.
Put these in a pan with a few tablespoons of melted butter. Cook briefly until the squid firms. (Ten or fifteen seconds, thirty at most.)
Add the squid to the simmering potato mixture.
Add a cup or two of small scallops, and a cup of small shrimp.
Cook a roux, equal parts butter and flour until the flour is golden-brown.
Add it to the simmering mixture and whisk to combine, and remove the heat.
Add 3/4 cup of heavy cream, or 1 1/2 cups of half and half.
Let stand for a bit, and serve.
Season with salt and pepper, and add a quarter cup of chopped fresh dill.
Simmer until warmed through again. Don’t let the scallops overcook.
Let cool slightly
A simple primer on cryptographic primitives
A field guide
Or “don’t trust anything that screws these up even slightly.”
Key
A private, hard to guess piece of information, meaningless on its own, but used to secure other pieces of information.
Public Key / Private Key
Specifically, these are keys with certain properties: They come as a pair, they’re usually a couple prime numbers (which are mathematically hard to factor, which is where their security comes from)
Things encrypted with one key can be decrypted with the other and vice versa.
Hash
Using a cryptographic hash function (which is often based on an encryption function, but not always) takes an often big piece of information and turns it into a fixed length token that represents it, in a hard to fake way. Even small changes will make a cryptographically strong hash function change its output entirely.
Some example hash functions: MD5, SHA1, SHA256, SHA512
Signature
The result of using a key and a hash function together on a piece of information to give some proof that the information wasn’t forged. If the key and signing algorithm used are public/private paired keys, then the public key can verify that the information was signed by the private key.
Certificate
A signature on a public key, and usually some ID information. If the certificate was signed by a trusted party (trust is a complicated thing, though) then there’s usually some assurance that the information signed by the the private key that matches the certificate is from a known source. Of course, can you spot a forged ID?
HMAC
A way of hashing information with a key securely to form a signature that can’t be altered. Turns out that if you just start with the key and add data to the end of it, then hash that, an attacker can keep adding things and keep running the hash function from where it left off and the signature will look valid. HMAC mixes the key with the information being signed in a way that prevents this.
Salt
When you’re using a hash to make information hard to brute-force, you make sure that an attacker can’t just build a list of all the likely things and see if you have them by adding randomness to the thing you’re hashing. Now, since this changes the hash value, you have to include it in a way that the thing comparing the hash can do the same way, so a salted hash often looks like data + salt
= $salt$HASH
. Usually this is combined with a very slow, hard to do hash function, so you can’t just whip through all the possibilities on a fast computer in a day or two. Computers keep getting faster, though…
TL;DR
Key = random unguessable; Key + hash = signature, signature + keypair = certificate; Hash + salt = hard to crack hash + salt.
Quote: Design Philosophies of Developer Tools
One of the nice things about Git is how its internals are both exposed for the world to see and thoroughly documented. We can easily write scripts to automate common tasks or create different workflows. With a bit more effort, we could even write new tools that integrate with the Git suite. These tools can do things that Git’s authors never intended, as long as they follow the documented repository structure. Git isn’t so much a version control system as the means to construct one.
And</>
All of the Ruby development tools have independent release cycles, and they don’t seem to plan or coordinate with one another in advance of each release. Integration testing is left up to the users.
Design Philosophies of Developer Tools (via Digital Digressions by Stuart Sierra)
Some very good thoughts.
Gluten-free Crullers
Boil 1 cup of water
Add 2 sticks of butter and let it melt completely
Remove the mixture from heat.
Add 1 cup of Glutino all-purpose gluten-free flour. Another mix with some bean flour might have better texture at the expense of flavor.
Add 1/2 teaspoon Xanthan Gum
Add 1 tablespoon tapioca flour
Beat and add three eggs, one at a time, incorporating completely.
The dough will start out the texture of mashed potatoes, but eventually become a soft, pliable consistency between dough and batter. Work the batter hard until it’s completely smooth.
Heat oil for deep frying to 375 °F.
Fill a pastry bag with the dough, cut a 1/2" hole for the tip. Squeeze sticks or curls into the hot oil carefully.
Fry until golden.
A great idea for SVG fonts
From this www-font posting by Adam Twardoch
INTRODUCTION
Obviously, SVG Fonts have some good and interesting concepts. One of
their advantages is that they can – at least in theory – freely
combine all aspects of SVG: multi-colored, multi-layered vector
graphics, and bitmaps.
However, SVG Fonts also have some serious drawbacks: while the glyph
definition using SVG is a great concept, all the other aspects of SVG
Fonts that make them work as a font, especially the character mapping,
access to alternate glyphs, and the layout behavior, are somewhat
under-defined and hard to implement. Therefore, it’s rather unlikely
that at any time, all OS and application vendors will agree on a good,
full implementation of SVG Fonts.
Therefore, I’d like to suggest a different path: place an SVG Font as a
table inside of an OpenType font*, and combine the power of both formats.
Batch extract audio from mp4 videos
for I in *.mp4; do ffmpeg -vn -acodec copy -i "$I" "$I.m4a"; done
Thoughts on a (maybe) sane build system.
I’ve been thinking about build systems a lot this week, thanks to V8’s terrible use of scons, its replacement, gyp, being a broken pile of steaming still, and everyone’s collective hatred of autoconf.
I think Guru (vaporware with a good idea) is onto something, though.
I think a lot of build systems are too process-focused, which is exactly the path that leads to platform dependence and the craziness that’s come before.
If each module declares what it should know about the process, that’s a start: main.c
knows that it’s the entry point of a program. It can say so. foofuncs.c
knows that it’s the implementation of the functions defined in foofuncs.h
, but it doesn’t know whether it’s destined for a static or dynamic library, or even just #include
ed into other code. It can declare the parts it knows about.
Then, there can be module-level declarations: “These things form a coherent library”, “These parts are required for feature frobnicate”, “this must be linked with library having function foo”
Then at the package level, one has to configure major options – are we installing in an application specific root, or a system-wide one? Are we building full-featured or light? Cover-your-ass static linking of everything for a build that works everywhere, or shared-everything to play nice with the specific system being installed.
What do you want a build system to do, and more, not do?
Oh, the unsuspected woes of server migration
Last night, I migrated The Internet Company’s servers from a Linux-VServer host to an OpenVZ host. It all went well, except one crazy detail.
The OpenVZ host runs on CentOS, and apparently its way of calling gettimeofday(2)
doesn’t agree with PLD’s glibc’s way. Specifically, the 9th bit of the resulting time is wrong … some of the time.
` with kernel.vsyscall64=0: 2011-05-09 17:14:48 -0600 1304982888 1001101110010000111010101101000
with kernel.vsyscall64=1: 2011-05-09 17:19:56 -0600 1304983196 1001101110010000111011010011100
with kernel.vsyscall64=0: 2011-05-09 17:15:04 -0600 1304982904 1001101110010000111010101111000 `
Fixed!
So what happens is that you get errors like Dovecot complaining of things like this:
May 9 15:36:31 host dovecot: pop3: Fatal: Time just moved backwards by 290 seconds. This might cause a lot of problems, so I'll just kill myself now. http://wiki2.dovecot.org/TimeMovedBackwards
since the mail server needs to know what time things arrived, and the date going backward is a sign that the clock is Not Reliable.
It also gives errors like this when things move forward again:
May 9 15:33:54 host dovecot: imap-login: Error: master(imap): Auth request timed out (received 0/12 bytes)
May 9 15:33:54 host dovecot: pop3-login: Error: master(pop3): Auth request timed out (received 0/12 bytes)
May 9 15:33:54 host dovecot: imap-login: Error: master(imap): Auth request timed out (received 0/12 bytes)
So the solution? set kernel.vsyscall64=0
. It fixes the mismatch between guest and host on OpenVZ, making the five-minute jump disappear. Just pop that line into /etc/sysctl.conf
and then apply it with sysctl -p
.