What slang can us older relatives use tonight to embarrass the teenagers while opening presents? [Published articles]

There’s New Proof Crispr Can Edit Genes Inside Human Bodies [Published articles]

The technique had largely been limited to editing patients’ cells in the lab. New research shows promise for treating diseases more directly.

Terrier: An Open-Source Tool for Identifying and Analyzing Container and Image Components [Published articles]

As part of our Blackhat Europe talk “Reverse Engineering and Exploiting Builds in the Cloud” we publicly released a new tool called Terrier.

Announcing Terrier: An open-source tool for identifying and analysing container and image components.

In this blog post, I am going to show you how Terrier can help you identify and verify container and image components for a wide variety of use-cases, be it from a supply-chain perspective or forensics perspective. Terrier can be found on Github https://github.com/heroku/terrier.

Containers and images

In this blog post, I am not going to go into too much detail about containers and images (you can learn more here) however it is important to highlight a few characteristics of containers and images that make them interesting in terms of Terrier. Containers are run from images and currently the Open Containers Initiative (OCI) is the most popular format for images. The remainder of this blog post refers to OCI images as images.

Essentially images are tar archives that container multiple tar archives and meta-information that represent the “layers” of an image. The OCI format of images makes images relatively simple to work with which makes analysis relatively simple. If you only had access to a terminal and the tar command, you could pretty much get what you need from the image’s tar archive.

When images are utilised at runtime for a container, their contents become the contents of the running container and the layers are essentially extracted to a location on the container’s runtime host. The container runtime host is the host that is running and maintaining the containers. This location is typically /var/lib/docker/overlay2/<containerID>/. This location contains a few folders of interest, particularly the "merged" folder. The "merged" folder contains the contents of the image and any changes that have occurred in the container since its creation. For example, if the image contained a location such as /usr/chris/stuff and after creating a container from this image I created a file called helloworld.txt at the location /usr/chris/stuff. This would result in the following valid path on the container runtime host /var/lib/docker/overlay2/<containerID>/merged/usr/chris/stuff/helloworld.txt.

What does Terrier do?

Now that we have a brief understanding of images and containers, we can look at what Terrier does. Often it is the case that you would like to determine if an image or container contains a specific file. This requirement may be due to a forensic analysis need or to identify and prevent a certain supply-chain attack vector. Regardless of the requirement, having the ability to determine the presence of a specific file in an image or container is useful.

Identifying files in OCI images

Terrier can be used to determine if a specific image contains a specific file. In order to do this, you need the following:

  1. An OCI Image i.e TAR archive
  2. A SHA256 hash of a specific file/s

The first point can be easily achieved with Docker by using the following command:

$ docker save imageid -o myImage.tar

The command above uses a Docker image ID which can be obtained using the following command:

$ docker images

Once you have your image exported as a tar archive, you will then need to establish the SHA256 hash of the particular file you would like to identify in the image. There are multiple ways to achieve this but in this example, we are going to use the hash of the Golang binary go1.13.4 linux/amd64 which can be achieved with following command on a Linux host:

$ cat /usr/local/go/bin/go | sha256sum

The command above should result in the following SHA256 hash: 82bce4b98d7aaeb4f841a36f7141d540bb049f89219f9e377245a91dd3ff92dd

Now that we have a hash, we can use this hash to determine if the Golang binary is in the image myImage.tar. To achieve this, we need to populate a configuration file for Terrier. Terrier makes use of YAML configuration files and below is our config file that we save as cfg.yml:

mode: image
image: myImage.tar

hashes:
    - hash: '82bce4b98d7aaeb4f841a36f7141d540bb049f89219f9e377245a91dd3ff92dd'

The config file above has multiple entries which allow us to specify the mode that Terrier will operate in and in this case, we are working with an image file (tar archive) so the mode is image. The image file we are working with is myImage.tar and the hash we are looking to identify is in the hashes list.

We are now ready to run Terrier and this can be done with the following command:

$ ./terrier

The command above should result in output similar to the following:

$ ./terrier 
[+] Loading config:  cfg.yml
[+] Analysing Image
[+] Docker Image Source:  myImage.tar
[*] Inspecting Layer:  34a9e0f17132202a82565578a3c2dae1486bb198cde76928c8c2c5c461e11ccf
[*] Inspecting Layer:  6539a80dd09da08132a525494ff97e92f4148d413e7c48b3583883fda8a40560
[*] Inspecting Layer:  6d2d61c78a65b6e6c82b751a38727da355d59194167b28b3f8def198cd116759
[!] Found file '6d2d61c78a65b6e6c82b751a38727da355d59194167b28b3f8def198cd116759/usr/local/go/bin/go' with hash: 82bce4b98d7aaeb4f841a36f7141d540bb049f89219f9e377245a91dd3ff92dd
[*] Inspecting Layer:  a6e646c34d2d2c2f4ab7db95e4c9f128721f63c905f107887839d3256f1288e1
[*] Inspecting Layer:  aefc8f0c87a14230e30e510915cbbe13ebcabd611e68db02b050b6ceccf9c545
[*] Inspecting Layer:  d4468fff8d0f28d87d48f51fc0a6afd4b38946bbbe91480919ebfdd55e43ce8c
[*] Inspecting Layer:  dbf9da5e4e5e1ecf9c71452f6b67b2b0225cec310a20891cc5dedbfd4ead667c

We have identified a file /usr/local/go/bin/go located at layer 6d2d61c78a65b6e6c82b751a38727da355d59194167b28b3f8def198cd116759 that has the same SHA256 hash as the one we provided. We now have verification that the image “myImage.tar” contains a file with the SHA256 hash we provided.

This example can be extended upon and you can instruct Terrier to search for multiple hashes. In this case, we are going to search for a malicious file. Recently a malicious Python library was identified in the wild and went by the name “Jeilyfish”. Terrier could be used to check if a Docker image of yours contained this malicious package. To do this, we can determine the SHA256 of one of the malicious Python files that contains the backdoor:

$ cat jeIlyfish-0.7.1/jeIlyfish/_jellyfish.py | sha256sum
cf734865dd344cd9b0b349cdcecd83f79a751150b5fd4926f976adddb93d902c

We then update our Terrier config to include the hash calculated above.

mode: image
image: myImage.tar

hashes:
    - hash: '82bce4b98d7aaeb4f841a36f7141d540bb049f89219f9e377245a91dd3ff92dd'
    - hash: 'cf734865dd344cd9b0b349cdcecd83f79a751150b5fd4926f976adddb93d902c'

We then run Terrier against and analyse the results:

$ ./terrier 
[+] Loading config:  cfg.yml
[+] Analysing Image
[+] Docker Image Source:  myImage.tar
[*] Inspecting Layer:  34a9e0f17132202a82565578a3c2dae1486bb198cde76928c8c2c5c461e11ccf
[*] Inspecting Layer:  6539a80dd09da08132a525494ff97e92f4148d413e7c48b3583883fda8a40560
[*] Inspecting Layer:  6d2d61c78a65b6e6c82b751a38727da355d59194167b28b3f8def198cd116759
[!] Found file '6d2d61c78a65b6e6c82b751a38727da355d59194167b28b3f8def198cd116759/usr/local/go/bin/go' with hash: 82bce4b98d7aaeb4f841a36f7141d540bb049f89219f9e377245a91dd3ff92dd
[*] Inspecting Layer:  a6e646c34d2d2c2f4ab7db95e4c9f128721f63c905f107887839d3256f1288e1
[*] Inspecting Layer:  aefc8f0c87a14230e30e510915cbbe13ebcabd611e68db02b050b6ceccf9c545
[*] Inspecting Layer:  d4468fff8d0f28d87d48f51fc0a6afd4b38946bbbe91480919ebfdd55e43ce8c
[*] Inspecting Layer:  dbf9da5e4e5e1ecf9c71452f6b67b2b0225cec310a20891cc5dedbfd4ead667c

The results above indicate that our image did not contain the malicious Python package.

There is no limit as to how many hashes you can search for however it should be noted that Terrier performs all its actions in-memory for performance reasons so you might hit certain limits if you do not have enough accessible memory.

Identifying and verifying specific files in OCI images

Terrier can also be used to determine if a specific image contains a specific file at a specific location. This can be useful to ensure that an image is using a specific component i.e binary, shared object or dependency. This can also be seen as “pinning” components by ensuring that you are images are using specific components i.e a specific version of cURL.

In order to do this, you need the following:

  1. An OCI Image i.e TAR archive
  2. A SHA256 hash of a specific file/s
  3. The path and name of the specific file/s

The first point can be easily achieved with Docker by using the following command:

$ docker save imageid -o myImage.tar

The command above utilises a Docker image id which can be obtained using the following command:

$ docker images

Once you have your image exported as a tar archive, you will need to determine the path of the file you would like to identify and verify in the image. For example, if we would like to ensure that our images are making use of a specific version of cURL, we can run the following commands in a container or some other environment that resembles the image.

$ which curl
/usr/bin/curl

We now have the path to cURL and can now generate the SHA256 of this instance of cURL because in this case, we trust this instance of cURL. We could determine the hash by other means for example many binaries are released with a corresponding hash from the developer which can be acquired from the developer’s website.

$ cat /usr/bin/curl | sha256sum 
9a43cb726fef31f272333b236ff1fde4beab363af54d0bc99c304450065d9c96

With this information, we can now populate our config file for Terrier:

mode: image
image: myImage.tar
files:
  - name: '/usr/bin/curl'
    hashes:
      - hash: '9a43cb726fef31f272333b236ff1fde4beab363af54d0bc99c304450065d9c96'

We’ve saved the above config as cfg.yml and when we run Terrier with this config, we get the following output:

$ ./terrier
[+] Loading config:  cfg.yml
[+] Analysing Image
[+] Docker Image Source:  myImage.tar
[*] Inspecting Layer:  34a9e0f17132202a82565578a3c2dae1486bb198cde76928c8c2c5c461e11ccf
[*] Inspecting Layer:  34a9e0f17132202a82565578a3c2dae1486bb198cde76928c8c2c5c461e11ccf
[*] Inspecting Layer:  6539a80dd09da08132a525494ff97e92f4148d413e7c48b3583883fda8a40560
[*] Inspecting Layer:  6539a80dd09da08132a525494ff97e92f4148d413e7c48b3583883fda8a40560
[*] Inspecting Layer:  6d2d61c78a65b6e6c82b751a38727da355d59194167b28b3f8def198cd116759
[*] Inspecting Layer:  6d2d61c78a65b6e6c82b751a38727da355d59194167b28b3f8def198cd116759
[*] Inspecting Layer:  a6e646c34d2d2c2f4ab7db95e4c9f128721f63c905f107887839d3256f1288e1
[*] Inspecting Layer:  a6e646c34d2d2c2f4ab7db95e4c9f128721f63c905f107887839d3256f1288e1
[*] Inspecting Layer:  aefc8f0c87a14230e30e510915cbbe13ebcabd611e68db02b050b6ceccf9c545
[*] Inspecting Layer:  aefc8f0c87a14230e30e510915cbbe13ebcabd611e68db02b050b6ceccf9c545
[*] Inspecting Layer:  d4468fff8d0f28d87d48f51fc0a6afd4b38946bbbe91480919ebfdd55e43ce8c
[*] Inspecting Layer:  d4468fff8d0f28d87d48f51fc0a6afd4b38946bbbe91480919ebfdd55e43ce8c
[*] Inspecting Layer:  dbf9da5e4e5e1ecf9c71452f6b67b2b0225cec310a20891cc5dedbfd4ead667c
[*] Inspecting Layer:  dbf9da5e4e5e1ecf9c71452f6b67b2b0225cec310a20891cc5dedbfd4ead667c
[!] All components were identified: (1/1)
[!] All components were identified and verified: (1/1)
$ echo $?
0

The output above indicates that the file /usr/bin/curl was successfully identified and verified, meaning that the image contained a file at the location /usr/bin/curl and that the SHA256 of that file matched the hash we provided in the config. Terrier also makes use of return codes and if we analyse the return code from the output above, we can see that the value is 0 which indicates a success. If Terrier cannot identify or verify all the provided files, a return code of 1 is returned which indicates a failure. The setting of return codes is particularly useful in testing environments or CI/CD environments.

We can also run Terrier with verbose mode enable to get more information:

$ ./terrier 
[+] Loading config:  cfg.yml
[+] Analysing Image
[+] Docker Image Source:  myImage.tar
[*] Inspecting Layer:  34a9e0f17132202a82565578a3c2dae1486bb198cde76928c8c2c5c461e11ccf
[*] Inspecting Layer:  6539a80dd09da08132a525494ff97e92f4148d413e7c48b3583883fda8a40560
        [!] Identified  instance of '/usr/bin/curl' at: 6539a80dd09da08132a525494ff97e92f4148d413e7c48b3583883fda8a40560/usr/bin/curl 
        [!] Verified matching instance of '/usr/bin/curl' at: 6539a80dd09da08132a525494ff97e92f4148d413e7c48b3583883fda8a40560/usr/bin/curl with hash: 9a43cb726fef31f272333b236ff1fde4beab363af54d0bc99c304450065d9c96
[*] Inspecting Layer:  6d2d61c78a65b6e6c82b751a38727da355d59194167b28b3f8def198cd116759
[*] Inspecting Layer:  a6e646c34d2d2c2f4ab7db95e4c9f128721f63c905f107887839d3256f1288e1
[*] Inspecting Layer:  aefc8f0c87a14230e30e510915cbbe13ebcabd611e68db02b050b6ceccf9c545
[*] Inspecting Layer:  d4468fff8d0f28d87d48f51fc0a6afd4b38946bbbe91480919ebfdd55e43ce8c
[*] Inspecting Layer:  dbf9da5e4e5e1ecf9c71452f6b67b2b0225cec310a20891cc5dedbfd4ead667c
[!] All components were identified: (1/1)
[!] All components were identified and verified: (1/1)

The output above provides some more detailed information such as which layer the cURL files was located at. If you wanted more information, you could enable the veryveryverbose option in the config file but beware, this is a lot of output and grep will be your friend.

There is no limit for how many hashes you can specify for a file. This can be useful for when you want to allow more than one version of a specific file i.e multiple versions of cURL. An example config of multiple hashes for a file might look like:

mode: image
image: myImage.tar
files:
  - name: '/usr/bin/curl'
    hashes:
      - hash: '9a43cb726fef31f272333b236ff1fde4beab363af54d0bc99c304450065d9c96'
      - hash: 'aefc8f0c87a14230e30e510915cbbe13ebcabd611e68db02b050b6ceccf9c545'
      - hash: '6d2d61c78a65b6e6c82b751a38727da355d59194167b28b3f8def198cd116759'
      - hash: 'd4468fff8d0f28d87d48f51fc0a6afd4b38946bbbe91480919ebfdd55e43ce8c'

The config above allows Terrier to verify if the identified cURL instance is one of the provided hashes. There is also no limit for the amount of files Terrier can attempt to identify and verify.

Terrier’s Github repo also contains a useful script called convertSHA.sh which can be used to convert a list of SHA256 hashes and filenames into a Terrier config file. This is useful when converting the output from other tools into a Terrier friendly format. For example, we could have the following contents of a file:

8946690bfe12308e253054ea658b1552c02b67445763439d1165c512c4bc240d ./bin/uname
6de8254cfd49543097ae946c303602ffd5899b2c88ec27cfcd86d786f95a1e92 ./bin/gzexe
74ff9700d623415bc866c013a1d8e898c2096ec4750adcb7cd0c853b4ce11c04 ./bin/wdctl
61c779de6f1b9220cdedd7dfee1fa4fb44a4777fff7bd48d12c21efb87009877 ./bin/dmesg
7bdde142dc5cb004ab82f55adba0c56fc78430a6f6b23afd33be491d4c7c238b ./bin/which
3ed46bd8b4d137cad2830974a78df8d6b1d28de491d7a23d305ad58742a07120 ./bin/mknod
e8ca998df296413624b2bcf92a31ee3b9852f7590f759cc4a8814d3e9046f1eb ./bin/mv
a91d40b349e2bccd3c5fe79664e70649ef0354b9f8bd4658f8c164f194b53d0f ./bin/chown
091abe52520c96a75cf7d4ff38796fc878cd62c3a75a3fd8161aa3df1e26bebd ./bin/uncompress
c5ebd611260a9057144fd1d7de48dbefc14e16240895cb896034ae05a94b5750 ./bin/echo
d4ba9ffb5f396a2584fec1ca878930b677196be21aee16ee6093eb9f0a93bf8f ./bin/df
5fb515ff832650b2a25aeb9c21f881ca2fa486900e736dfa727a5442a6de83e5 ./bin/tar
6936c9aa8e17781410f286bb1cbc35b5548ea4e7604c1379dc8e159d91a0193d ./bin/zforce
8d641329ea7f93b1caf031b70e2a0a3288c49a55c18d8ba86cc534eaa166ec2e ./bin/gzip
0c1a1f53763ab668fb085327cdd298b4a0c1bf2f0b51b912aa7bc15392cd09e7 ./bin/su
20c358f7ee877a3fd2138ecce98fada08354810b3e9a0e849631851f92d09cc4 ./bin/bzexe
01764d96697b060b2a449769073b7cf2df61b5cb604937e39dd7a47017e92ee0 ./bin/znew
0d1a106dc28c3c41b181d3ba2fc52086ede4e706153e22879e60e7663d2f6aad ./bin/login
fb130bda68f6a56e2c2edc3f7d5b805fd9dcfbcc26fb123a693b516a83cfb141 ./bin/dir
0e7ca63849eebc9ea476ea1fefab05e60b0ac8066f73c7d58e8ff607c941f212 ./bin/bzmore
14dc8106ec64c9e2a7c9430e1d0bef170aaad0f5f7f683c1c1810b466cdf5079 ./bin/zless
9cf4cda0f73875032436f7d5c457271f235e59c968c1c101d19fc7bf137e6e37 ./bin/chmod
c5f12f157b605b1141e6f97796732247a26150a0a019328d69095e9760b42e38 ./bin/sleep
b9711301d3ab42575597d8a1c015f49fddba9a7ea9934e11d38b9ff5248503a8 ./bin/zfgrep
0b2840eaf05bb6802400cc5fa793e8c7e58d6198334171c694a67417c687ffc7 ./bin/stty
d9393d0eca1de788628ad0961b74ec7a648709b24423371b208ae525f60bbdad ./bin/bunzip2
d2a56d64199e674454d2132679c0883779d43568cd4c04c14d0ea0e1307334cf ./bin/mkdir
1c48ade64b96409e6773d2c5c771f3b3c5acec65a15980d8dca6b1efd3f95969 ./bin/cat
09198e56abd1037352418279eb51898ab71cc733642b50bcf69d8a723602841e ./bin/true
97f3993ead63a1ce0f6a48cda92d6655ffe210242fe057b8803506b57c99b7bc ./bin/zdiff
0d06f9724af41b13cdacea133530b9129a48450230feef9632d53d5bbb837c8c ./bin/ls
da2da96324108bbe297a75e8ebfcb2400959bffcdaa4c88b797c4d0ce0c94c50 ./bin/zegrep

The file contents above are trusted SHA256 hashes for specific files. If we would like to use this list for ensuring that a particular image is making use of the files listed above, we can do the following:

$ ./convertSHA.sh trustedhashes.txt terrier.yml

The script above takes the input file trustedhashes.txt which contains our trusted hashes listed above and converts them into a Terrier friendly config file called terrier.yml which looks like the following:

mode: image
image: myImage.tar
files:
  - name: '/bin/uname'
    hashes:
       - hash: '8946690bfe12308e253054ea658b1552c02b67445763439d1165c512c4bc240d'
  - name: '/bin/gzexe'
    hashes:
       - hash: '6de8254cfd49543097ae946c303602ffd5899b2c88ec27cfcd86d786f95a1e92'
  - name: '/bin/wdctl'
    hashes:
       - hash: '74ff9700d623415bc866c013a1d8e898c2096ec4750adcb7cd0c853b4ce11c04'
  - name: '/bin/dmesg'
    hashes:
       - hash: '61c779de6f1b9220cdedd7dfee1fa4fb44a4777fff7bd48d12c21efb87009877'
  - name: '/bin/which'
    hashes:
       - hash: '7bdde142dc5cb004ab82f55adba0c56fc78430a6f6b23afd33be491d4c7c238b'
  - name: '/bin/mknod'

The config file terrier.yml is ready to be used:

$ ./terrier -cfg=terrier.yml
[+] Loading config:  terrier.yml
[+] Analysing Image
[+] Docker Image Source:  myImage.tar
[*] Inspecting Layer:  34a9e0f17132202a82565578a3c2dae1486bb198cde76928c8c2c5c461e11ccf
[*] Inspecting Layer:  6539a80dd09da08132a525494ff97e92f4148d413e7c48b3583883fda8a40560
[*] Inspecting Layer:  6d2d61c78a65b6e6c82b751a38727da355d59194167b28b3f8def198cd116759
[*] Inspecting Layer:  a6e646c34d2d2c2f4ab7db95e4c9f128721f63c905f107887839d3256f1288e1
[*] Inspecting Layer:  aefc8f0c87a14230e30e510915cbbe13ebcabd611e68db02b050b6ceccf9c545
[*] Inspecting Layer:  d4468fff8d0f28d87d48f51fc0a6afd4b38946bbbe91480919ebfdd55e43ce8c
[*] Inspecting Layer:  dbf9da5e4e5e1ecf9c71452f6b67b2b0225cec310a20891cc5dedbfd4ead667c
[!] Not all components were identifed: (4/31)
[!] Component not identified:  /bin/uncompress
[!] Component not identified:  /bin/bzexe
[!] Component not identified:  /bin/bzmore
[!] Component not identified:  /bin/bunzip2
$ echo $?
1

As we can see from the output above, Terrier was unable to identify 4/31 of the components provided in the config. The return code is also 1 which indicates a failure. If we were to remove the components that are not in the provided image, the output from the previous command would look like the following:

$ ./terrier -cfg=terrier.yml
[+] Loading config: terrier.yml
[+] Analysing Image
[+] Docker Image Source: myImage.tar
[*] Inspecting Layer: 34a9e0f17132202a82565578a3c2dae1486bb198cde76928c8c2c5c461e11ccf
[*] Inspecting Layer: 6539a80dd09da08132a525494ff97e92f4148d413e7c48b3583883fda8a40560
[*] Inspecting Layer: 6d2d61c78a65b6e6c82b751a38727da355d59194167b28b3f8def198cd116759
[*] Inspecting Layer: a6e646c34d2d2c2f4ab7db95e4c9f128721f63c905f107887839d3256f1288e1
[*] Inspecting Layer: aefc8f0c87a14230e30e510915cbbe13ebcabd611e68db02b050b6ceccf9c545
[*] Inspecting Layer: d4468fff8d0f28d87d48f51fc0a6afd4b38946bbbe91480919ebfdd55e43ce8c
[*] Inspecting Layer: dbf9da5e4e5e1ecf9c71452f6b67b2b0225cec310a20891cc5dedbfd4ead667c
[!] All components were identified: (27/27)
[!] Not all components were verified: (26/27)
[!] Component not verified: /bin/cat
[!] Component not verified: /bin/chmod
[!] Component not verified: /bin/chown
[!] Component not verified: /bin/df
[!] Component not verified: /bin/dir
[!] Component not verified: /bin/dmesg
[!] Component not verified: /bin/echo
[!] Component not verified: /bin/gzexe
[!] Component not verified: /bin/gzip
[!] Component not verified: /bin/login
[!] Component not verified: /bin/ls
[!] Component not verified: /bin/mkdir
[!] Component not verified: /bin/mknod
[!] Component not verified: /bin/mv
[!] Component not verified: /bin/sleep
[!] Component not verified: /bin/stty
[!] Component not verified: /bin/su
[!] Component not verified: /bin/tar
[!] Component not verified: /bin/true
[!] Component not verified: /bin/uname
[!] Component not verified: /bin/wdctl
[!] Component not verified: /bin/zdiff
[!] Component not verified: /bin/zfgrep
[!] Component not verified: /bin/zforce
[!] Component not verified: /bin/zless
[!] Component not verified: /bin/znew
$ echo $?
1

The output above indicates that Terrier was able to identify all the components provided but many were not verifiable, the hashes did not match and once again, the return code is 1 to indicate this failure.

Identifying files in containers

The previous sections focused on identifying files in images, which can be referred to as a form of “static analysis,” however it is also possible to perform this analysis to running containers. In order to do this, you need the following:

  1. Location of the container’s merged folder
  2. A SHA256 hash of a specific file/s

The merged folder is Docker specific, in this case, we are using it because this is where the contents of the Docker container reside, this might be another location if it were LXC.

The location of the container’s merged folder can be determined by running the following commands. First obtain the container’s ID:

$ docker ps
CONTAINER ID        IMAGE                    COMMAND               CREATED             STATUS              PORTS               NAMES
b9e676fd7b09        golang                   "bash"                20 hours ago        Up 20 hours                             cocky_robinson

Once you have the container’s ID, you can run the following command which will help you identify the location of the container’s merged folder on the underlying host.

$ docker exec b9e676fd7b09 mount | grep diff
overlay on / type overlay (rw,relatime,lowerdir=/var/lib/docker/overlay2/l/7ZDEFE6PX4C3I3LGIGGI5MWQD4:
/var/lib/docker/overlay2/l/EZNIFFIXOVO2GIT5PTBI754HC4:/var/lib/docker/overlay2/l/UWKXP76FVZULHGRKZMVYJHY5IK:
/var/lib/docker/overlay2/l/DTQQUTRXU4ZLLQTMACWMJYNRTH:/var/lib/docker/overlay2/l/R6DE2RY63EJABTON6HVSFRFICC:
/var/lib/docker/overlay2/l/U4JNTFLQEKMFHVEQJ5BQDLL7NO:/var/lib/docker/overlay2/l/FEBURQY25XGHJNPSFY5EEPCFKA:
/var/lib/docker/overlay2/l/ICNMAZ44JY5WZQTFMYY4VV6OOZ,
upperdir=/var/lib/docker/overlay2/04f84ddd30a7df7cd3f8b1edeb4fb89d476ed84cf3f76d367e4ebf22cd1978a4/diff,
workdir=/var/lib/docker/overlay2/04f84ddd30a7df7cd3f8b1edeb4fb89d476ed84cf3f76d367e4ebf22cd1978a4/work)          

From the results above, we are interested in two entries, upperdir and workdir because these two entries will provide us with the path to the container’s merged folder. From the results above, we can determine that the container’s merged directory is located at /var/lib/docker/overlay2/04f84ddd30a7df7cd3f8b1edeb4fb89d476ed84cf3f76d367e4ebf22cd1978a4/ on the underlying host.

Now that we have the location, we need some files to identify and in this case, we are going to reuse the SHA256 hashes from the previous section. Let’s now go ahead and populate our Terrier configuration with this new information.

mode: container
path: merged
#image: myImage.tar

hashes:
    - hash: '82bce4b98d7aaeb4f841a36f7141d540bb049f89219f9e377245a91dd3ff92dd'
    - hash: 'cf734865dd344cd9b0b349cdcecd83f79a751150b5fd4926f976adddb93d902c'

The configuration above shows that we have changed the mode from image to container and we have added the path to our merged folder. We have kept the two hashes from the previous section.

If we run Terrier with this configuration from the location /var/lib/docker/overlay2/04f84ddd30a7df7cd3f8b1edeb4fb89d476ed84cf3f76d367e4ebf22cd1978a4/, we get the following output:

$ ./terrier
[+] Loading config: cfg.yml
[+] Analysing Container
[!] Found matching instance of '82bce4b98d7aaeb4f841a36f7141d540bb049f89219f9e377245a91dd3ff92dd' at: merged/usr/local/go/bin/go with hash:82bce4b98d7aaeb4f841a36f7141d540bb049f89219f9e377245a91dd3ff92dd

From the output above, we know that the container (b9e676fd7b09) does not contain the malicious Python package but it does contain an instance of the Golang binary which is located at merged/usr/local/go/bin/go.

Identifying and verifying specific files in containers

And as you might have guessed, Terrier can also be used to verify and identify files at specific paths in containers. To do this, we need the following:

  1. Location of the container’s merged folder
  2. A SHA256 hash of a specific file/s
  3. The path and name of the specific file/s

The points above can be determined using the same procedures described in the previous sections. Below is an example Terrier config file that we could use to identify and verify components in a running container:

mode: container
path: merged
verbose: true
files:
  - name: '/usr/bin/curl'
    hashes:
      - hash: '9a43cb726fef31f272333b236ff1fde4beab363af54d0bc99c304450065d9c96'
  - name: '/usr/local/go/bin/go'
    hashes:
      - hash: '82bce4b98d7aaeb4f841a36f7141d540bb049f89219f9e377245a91dd3ff92dd'

If we run Terrier with the above config, we get the following output:

$ ./terrier
[+] Loading config: cfg.yml
[+] Analysing Container
[!] Found matching instance of '/usr/bin/curl' at: merged/usr/bin/curl with hash:9a43cb726fef31f272333b236ff1fde4beab363af54d0bc99c304450065d9c96
[!] Found matching instance of '/usr/local/go/bin/go' at: merged/usr/local/go/bin/go with hash:82bce4b98d7aaeb4f841a36f7141d540bb049f89219f9e377245a91
dd3ff92dd
[!] All components were identified: (2/2)
[!] All components were identified and verified: (2/2)
$ echo $?
0

From the output above, we can see that Terrier was able to successfully identify and verify all the files in the running container. The return code is also 0 which indicates a successful execution of Terrier.

Using Terrier with CI/CD

In addition to Terrier being used as a standalone CLI tool, Terrier can also be integrated easily with existing CI/CD technologies such as GitHub Actions and CircleCI. Below are two example configurations that show how Terrier can be used to identify and verify certain components of Docker files in a pipeline and prevent the pipeline from continuing if all verifications do not pass. This can be seen as an extra mitigation for supply-chain attacks.

Below is a CircleCI example configuration using Terrier to verify the contents of an image.

version: 2
jobs:
build:
  machine: true
  steps:
    - checkout
    - run:
       name: Build Docker Image
       command: |
             docker build -t builditall .
    - run:
       name: Save Docker Image Locally
       command: |
             docker save builditall -o builditall.tar
    - run:
       name: Verify Docker Image Binaries
       command: |
             ./terrier

Below is a Github Actions example configuration using Terrier to verify the contents of an image.

name: Go
on: [push]
jobs:
build:
  name: Build
  runs-on: ubuntu-latest
  steps:

  - name: Get Code
    uses: actions/checkout@master
  - name: Build Docker Image
    run: |
      docker build -t builditall .
  - name: Save Docker Image Locally
    run: |
      docker save builditall -o builditall.tar
  - name: Verify Docker Image Binaries
    run: |
      ./terrier

Conclusion

In this blog post, we have looked at how to perform multiple actions on Docker (and OCI) containers and images via Terrier. The actions performed allowed us to identify specific files according to their hashes in images and containers. The actions performed have also allowed us to identify and verify multiple components in images and containers. These actions performed by Terrier are useful when attempting to prevent certain supply-chain attacks.

We have also seen how Terrier can be used in a DevOps pipeline via GitHub Actions and CircleCI.

Learn more about Terrier on GitHub at https://github.com/heroku/terrier.

They forged enduring friendships from massive, same-name group chats [Published articles]


Last month, journalist Matt Cohen tweeted about his years-long Instagram group chat comprised of fellow Matt Cohens, which he calls “the most wholesome thing I’m a part of.”

In the chat, one Matt Cohen shared that he “had [his] first day of college classes today,” to which a Matt Cohen responded “Nice. Just started my first job. Real world is brutal enjoy college man.”

“Got married!” and “Just started my dream job!” chimed in fellow Matt Cohens. Another Matt Cohen announced he had launched a weed brand. The Matt Cohens, who have turned a shared name into an informal online club, planned a Zoom Happy Hour to catch up.

Your name clones usually lurk around you like a shadow. You get their junk mail, their emails, their Google results; glimpses of their intimate moments via their digital ephemera. They are strangers — but they don’t have to be.

Hello, our name is...

Around the world, people are maintaining multigenerational, global friendships with their same-named counterparts — Jake Wright, William Hodgson, Jordan DaSilva, and Josh Brown, to name a few. Sometimes, name twins commiserate about shared experiences: a sixteen-member Council of Aaron Johnson chat laments about the viral Key and Peele sketch that introduced the now-inescapable A-A-Ron nickname. Perhaps the best, or at least the most publicized, example of same-name camaraderie is the Josh Fight, when a group chat of Josh Swains organized an April 2021 meeting in Lincoln, Nebraska to fight for the “right” to the name. More than 900 Joshes showed up.

The Paul O’Sullivan Band has four members with one thing in common: the name Paul O’Sullivan. The quartet materialized after Baltimore Paul started “indiscriminately adding other Paul O’Sullivans on Facebook” and realized that a few different Paul O’Sullivans were musicians. These days, a quartet of Paul O’Sullivans, who hail form from Baltimore, Rotterdam, Manchester, and Pennsylvania, have come together to form a bona fide musical group.

Since its early days, the social internet has been lauded as a way for niche interest groups to connect, and name twins are no exception. A chat titled “Council of Bens” hosts 2500 Benjamins and Bens, and when one Ben caught wind of a similar group chat of Sydneys, he created a chat just for people named Sydney or Ben, which has been going strong for months. Chris Lenaghan added 7 other Chris Lenaghans to a chat, and soon he had same-name friends from Ohio to Belfast to Birmingham. In a Josh Kaplan group chat on Twitter, fellow Josh Kaplans use the chat to congratulate each other on achievements and awards: “A win for one JK is a win for all.”

Samuel Stewart, a 19-year-old Exeter student living in London, formed an Instagram chat of fellow Samuel Stewarts after reading about the Josh Fight. For a few weeks, they chatted about their days; older Sam Stewarts gave advice to younger Sam Stewerts. “They seemed to take me under their wing as if I were a younger version of them,” said a 19-year-old Sam Stewert when we talked on the phone. But the chat went awry when one Samuel Stewert started asking for money. “I felt a bond with the fellow Samuel Stewarts, but the name connection wasn’t quite strong enough for me to start giving away my college fund,” Sam told me.

They seemed to take me under their wing as if I were a younger version of them.”

The chats aren’t strictly social — sometimes, they’re the most practical way to sort through same-name mixups. Will Packer, a strategist in New York, recently used the Will Packer chat to see if any of his name brothers had been contributing to his inbox clutter. “Any of you from Queensland?” he asked. “Someone tried to create a PlayStation account with my email.”

College student Nolen Young says, “I once created a Facebook Messenger group chat with everyone I could find on Facebook with my exact same name, spelling and all. There were only two other people. One of them considered giving me a job, and the other was an old man who started commenting on all my photos. I've messaged the former a few times because he owns every domain name and email I've ever wanted, and he keeps telling me I can only have them when he dies.”

It’s easier than ever to connect with same-name pals today, but the uncanny allure of name clones predates social media. Tahnee Gehm, an artist and animator based in L.A., organized a Web 1.0 catalog of Tahnees when she was a teenager.

“My dad was into computers and he got me a URL with my name,” she says. “I built an atrocious ‘90s website in 2001 as an eighth grader, and I started getting messages from girls all over the world named Tahnee.”

To catalog her new pen pals, she created a “Hall of Tahnees” webpage with a photo, bio, and hometown for every Tahnee she could find. The site’s “Tahnee-only area” was a “weird, unique club.” Once, she says, a singer from the band Hanson used the website to track down a girl named Tahnee he’d met at a concert. And the Tahnee bond has lasted decades: Tahnee Gehm has maintained a long-distance friendship with Tahneé Engelen since they were in high school. A few years ago, Gehm spent two weeks visiting Engelen in Paris, where she works as a neurobiologist.

“It’s nice to know that my name buddy is living my alternate life and absolutely killing it,” she told Input over the phone.

Sometimes, all it takes to spark a friendship is a similar email address. Seth Capron met an older Seth Capron after noticing their similar interests based on the emails he mistakenly received — soon, they realized their physical resemblance, too. These days, the older Seth jokes that he could pass on his career. “I was actually considering that as I move into retirement, the Younger could just carry on in my former role of Seth Capron, affordable housing consultant,” said “Seth the Older.”

Name buddies sometimes have a parasocial relationship with each other’s digital footprint. As a kid, Chris Lenaghan found online videos of a different Chris Lenaghan doing wheelies and “cool BMX shit” and immediately told all his friends that it was him in the videos. Years later, thanks to a big group chat, Chris Lenaghan met the BMX trickster, who he now calls “Ohio Chris,” and they ended up becoming close friends.

The chats don’t always advance beyond acquaintanceship, though. Evan Quigley, a University of Florida student, says that the Evan Quigley group chat is “more like a running joke than true friendship.” (The Evan Quigleys, bonded by name alone, proclaim unconditional public support for one another by commenting “way to go, Evan Quigley” on each other’s posts).

People with uncommon first names can bond over shared experiences — mispronunciations, playground taunts, and misspellings. More than a dozen Zaviens have come together via Snapchat. “None of us had ever talked to another Zavien,” one Zavien told Input. And a 14-member-strong “Council of Ethyns” chat, which started on Instagram in 2019, is mostly dedicated to tongue-in-cheek malice toward Ethans (with an “a”). They also just pop in the chat to say “love you Ethyn” a lot.

Still, the unlikely connections evoke nostalgia for a simpler internet, less cluttered with surveillance and corporate interests, where people went to meet new friends. Occasionally, wholesome chance online encounters remain. “Text door neighbors,” for example, or people with phone numbers one digit apart, show how easy it is to stumble upon an unlikely friend. Most notably in the wrong-number-gone-right stories, the duo Wanda and Jamal, whose viral wrong number ordeal has led to a six-year-long-and-counting Thanksgiving tradition, is now set to be featured in an upcoming Netflix movie.

It’s a big world out there — lots of Matt Cohens, more Alex Stewarts, and even more James Smiths — and your name buddies have never been easier to befriend. And I think that’s beautiful.

But just because we’re psychologically inclined to like our own name, doesn’t mean you’ll have a guaranteed connection with your name clones. Just ask Kelly Hildebrandt and Kelly Hildebrandt, the couple that tied the knot a year after they’d met when name-searching on Facebook and then, four years later, called off the marriage due to irreconcilable differences. It’s not all in a name.

Ursula Vernon Creates Hopeful Post-Apocalyptic Comic With Help of Image-Generating AI [Published articles]

Author and artist Ursula Vernon is not one to stay idle, and a few days ago she shared a hopeful albeit bittersweet comic about nature coming back after some unmentioned apocalyptic event. The story is a lovely one, and while Vernon can certainly draw, she decided for this project to use Midjourney, an AI program that creates images from textual descriptions.

Buy it Now

Vernon posted the mini-comic on Twitter (the beginning of which we will also embed at the end of this post), and also shared her process using Midjourney. Read on if you care to learn more about that, and if that doesn’t interest you, feel free to scroll to the bottom of the post to read her mini-comic, “A Different Aftermath.”

According to her thread, Vernon said she started her Midjourney journey in July. “I will walk through fire not to have to draw buildings,” she wrote. “Once I saw what [Midjourney] was capable of, the question was ‘Could the computer handle that for me?’”

In July, the answer was basically no. Vernon revisited the program in late August when it apparently got an upgrade, however, and had better luck. The results still needed some touching up but that process took her much less time and effort than before. She also knew that her story played to the AI’s strengths—silhouettes of people rather than full images, and that there weren’t multiple shots of the same place.

You can see the results below and now appreciate it on its own, as well as with the knowledge of how Vernon created it.

I made another weird little comic thing, hopeful and a little bittersweet, about conservation after the apocalypse. A topic near and dear to my heart, Lord knows. (Technical notes in another thread, linked at the end.) pic.twitter.com/weAcMbbu4Q

— Kingfisher & Wombat (@UrsulaV) September 10, 2022

Patagonia Founder Gives Away the Company To Fight Climate Change [Published articles]

A half century after founding the outdoor apparel maker Patagonia, Yvon Chouinard, the eccentric rock climber who became a reluctant billionaire with his unconventional spin on capitalism, has given the company away. The New York Times reports: Rather than selling the company or taking it public, Mr. Chouinard, his wife and two adult children have transferred their ownership of Patagonia, valued at about $3 billion, to a specially designed trust and a nonprofit organization. They were created to preserve the company's independence and ensure that all of its profits -- some $100 million a year -- are used to combat climate change and protect undeveloped land around the globe. The unusual move comes at a moment of growing scrutiny for billionaires and corporations, whose rhetoric about making the world a better place is often overshadowed by their contributions to the very problems they claim to want to solve. At the same time, Mr. Chouinard's relinquishment of the family fortune is in keeping with his longstanding disregard for business norms, and his lifelong love for the environment. "Hopefully this will influence a new form of capitalism that doesn't end up with a few rich people and a bunch of poor people,â Mr. Chouinard, 83, said in an exclusive interview. "We are going to give away the maximum amount of money to people who are actively working on saving this planet." Patagonia will continue to operate as a private, for-profit corporation based in Ventura, Calif., selling more than $1 billion worth of jackets, hats and ski pants each year. But the Chouinards, who controlled Patagonia until last month, no longer own the company. In August, the family irrevocably transferred all the company's voting stock, equivalent to 2 percent of the overall shares, into a newly established entity known as the Patagonia Purpose Trust. The trust, which will be overseen by members of the family and their closest advisers, is intended to ensure that Patagonia makes good on its commitment to run a socially responsible business and give away its profits. Because the Chouinards donated their shares to a trust, the family will pay about $17.5 million in taxes on the gift. The Chouinards then donated the other 98 percent of Patagonia, its common shares, to a newly established nonprofit organization called the Holdfast Collective, which will now be the recipient of all the company's profits and use the funds to combat climate change. Because the Holdfast Collective is a 501(c)(4), which allows it to make unlimited political contributions, the family received no tax benefit for its donation. Mr. Chouinard is certainly not like most ultra successful entrepreneurs today. The report notes that he "wears raggedy old clothes, drives a beat up Subaru and splits his time between modest homes in Ventura and Jackson, Wyo." He also doesn't own a computer or a cellphone. When the company's sales soared and Mr. Chouinard's net worth continued to climb, it made him uncomfortable because he abhors excessive wealth. "I was in Forbes magazine listed as a billionaire, which really, really pissed me off," he said. "I don't have $1 billion in the bank. I don't drive Lexuses." This ranking, along with the Covid-19 pandemic, "heped set in motion a process that would unfold over the past two years, and ultimately lead to the Chouinards giving away the company," the Times reports.

Read more of this story at Slashdot.

Baba’s Pantry in Kansas City named one of the country's 10 best new restaurants [Published articles]

Since opening in July 2021, the Palestinian restaurant and deli has quickly become a local favorite. This week, food magazine Bon Appétit ranked it among the very best in the country, thanks to its hummus and how it honors the Kamal family’s “Palestinian homeland and their life in the diaspora.”

Study shows nanoplastics can travel from plants to insects to fish [Published articles]

Polystyrene and PVC nanoparticles were found in roach fish, after those fish ate fly larvae which fed on lettuce plants grow in soil containing the particles

We already know that tiny waterborne plastic waste particles can enter the bodies of fish, which are then consumed by humans. New research, however, shows that such particles can enter the food chain via another route, by traveling from the land, through plants to insects to fish.

Continue Reading

Category: Environment, Science

Tags: University of Eastern Finland, Ocean plastics, Pollution, Fish, Insect, Plants

Interstellar Voyager [Published articles]

Voyager 1 and Voyager 2 were launched in 1977 on a Voyager 1 and Voyager 2 were launched in 1977 on a


FCC Has Obtained Detailed Broadband Maps From ISPs for the First Time Ever [Published articles]

The Federal Communications Commission has collected precise broadband availability information from Internet service providers for the first time and aims to release a first draft of a new broadband map in November, FCC Chairwoman Jessica Rosenworcel wrote Friday. From a report: The FCC last week "completed the first filing window for submitting data on where broadband service is and is not available," a milestone in the years-long process of creating an accurate US broadband map, she wrote. "For the first time ever, we have collected extensive location-by-location data on precisely where broadband services are available, and now we are ready to get to work and start developing new and improved broadband maps." The resulting map should show whether fixed broadband service is available at each residence or business location. The FCC's inaccurate broadband maps have long made it difficult to distribute deployment grants where they're needed most. Current maps are based on the Form 477 data-collection program in which ISPs report whether they offer service in each census block, which essentially lets ISPs count an entire census block as served even if it can serve just one home in the area. The new, more accurate maps will be used to help distribute $42.45 billion from the Broadband Equity, Access, and Deployment program created by Congress in the Infrastructure Investment and Jobs Act.

Read more of this story at Slashdot.

Missouri’s school funding system undermines its own goals for equity, experts say [Published articles]

Students walk away from a school bus

Missouri’s school funding strategy recognizes that some children and communities need more financial support to meet education standards.  

It directs extra funds to districts that have a harder time raising local property taxes, and to children who have special needs, are learning English or are living in poverty. 

But experts say while the system has good intentions, the devil is in the details. 

Parts of Missouri’s funding strategies undermine its equity goals. School finance researchers named issues that include: 

  • A method of counting enrollment that penalizes higher-poverty schools that struggle with attendance. 
  • A “threshold” system that means some districts don’t get additional funding for children with greater needs.
  • Funding levels based on outdated property values that lead districts with high economic growth to receive more funding than the formula is supposed to provide. 
  • More reliance on local funding — which particularly burdens low-income districts —- compared with nearly all other states.

Overall, this leads to a situation where some districts receive state aid they don’t need while others are stretched thin as they attempt to serve children who need more resources.  

“We have a large portion of students across the state who experience some form of economic disadvantage,” said Cameron Anglum, an assistant professor of education, policy and equity at the Saint Louis University School of Education

“It’s really important that the state funding formula serve those kids effectively, particularly those kids that live in districts that don’t have the local property wealth … to provide an adequate education.”

What makes a good school funding system

Bruce Baker, a professor and chair of the department of teaching and learning at the University of Miami School of Education and Human Development, said the main goals of a school finance formula should be adequacy and equity.

Adequacy means there’s enough funding for the school to meet certain goals. Equity acknowledges that some students or schools may require greater funding to meet those standards. 

To illustrate, Baker referred to the School Finance Indicators Database run by the Albert Shanker Institute and Rutgers Graduate School of Education. 

The database calculates that in 2019, the latest data available, the 20% of districts with the highest poverty rates in Missouri needed nearly $12,000 more per student to reach national average test scores than the 20% of districts with the lowest poverty rates

Instead, the database shows students in highest-poverty districts were receiving only about $1,000 more than students in the most affluent schools.  

Baker, one of the main researchers for the database, said the numbers are based on a statistical model that uses data on student characteristics, hiring costs and district size to calculate necessary funding levels, which are different in each state. 

James Shuls, an associate professor of educational leadership and policy studies at the University of Missouri-St. Louis, said school funding “reflects the values of people” and appropriate levels should be determined through the political process.  

Shuls said he personally values choice, equity and efficiency in education funding. He previously worked for the Show-Me Institute, where he authored a Missouri school finance formula primer. The institute is a think tank “dedicated to promoting free markets and individual liberty” and supportive of policies that increase “school choice.”

An ideal funding formula should be “dynamic,” said Shuls, reflecting changing local resources and specific student needs to better promote equity. 

Instead, Missouri’s formula reflects outdated property values and school funding levels, Shuls said.

How Missouri’s school funding formula works — and doesn’t work 

Missouri’s school funding formula starts with an “adequacy target,” the amount of money needed to educate a single student. It multiplies that number based on student attendance, area cost of living and, in some cases, student characteristics that might require extra funds such as disability or learning English. 

The formula then factors in how much funding districts can raise from local property taxes.

Anglum, the SLU professor, said one equity challenge is that the state doesn’t manage a majority of the funding that goes to schools. 

Some state funding also goes through programs that don’t have the same equity focus as the main formula.

Missouri ranks 47th out of 50 states when it comes to the percentage of school funding that comes from the state. When all sources — local, state and federal — are combined, 2018-19 data from the National Center for Education Statistics shows that K-12 per-student spending in Missouri ranks 32nd in the nation. 

Additionally, 2021 data from the National Education Association, a prominent teachers union, shows that not counting the District of Columbia, Missouri has the second-highest percentage of funding coming from local sources and the smallest percentage coming from state sources. 

“When we are relying predominantly on local resources in order to fund education, higher-wealth districts are going to win out and lower-wealth districts are going to lose out,” Anglum said. 

Traci Gleason, vice president for external affairs at the Missouri Budget Project, said lower state funding can cause localities with fewer resources to choose between underfunding services — including education — or imposing burdensome levels of property and sales taxes.  

‘No excuse’ to use average daily attendance

When legislators reformed school finance in 2005, they also included “hold harmless” provisions to ensure no district would receive less state money under the new formula. 

Shuls said that was a sensible way to prevent abrupt funding dips for some districts under the new system. But the “hold harmless” provisions didn’t phase out, meaning many districts are still being funded at outdated levels instead of updated, equitable ones.

When the formula calculates districts’ ability to raise local property taxes, it’s using property values that are now more than 16 years old. That means the state is giving districts with growing property values more funds than they need to meet targets, instead of distributing that money in other ways. 

Baker, the University of Miami professor with experience in Missouri and Kansas, said Missouri’s “hold harmless” provisions aren’t even the biggest factor that prevents the state from having a “progressive” funding system. (In this case, “progressive” means districts with greater need spend more money per student.) 

Hold harmless provisions tend to partially undo reforms, he agreed. “But I’m not convinced that any of the changes they were making would have very aggressively moved it in the right direction anyway.”

Instead, he said the state’s method of calculating attendance financially penalizes districts that most need support. 

Missouri calculates the number of students in each district by using the average daily attendance instead of the total number of students. 

That means a school with an 80% attendance rate on the average day could see its funding cut by 20% compared to an otherwise identical school with perfect attendance. 

Baker said that’s especially problematic when it comes to equity because schools with lower attendance rates tend to have higher rates of students living in poverty. 

“It’s been explained to policymakers in every damn state that it is discriminatory and erases any need adjustment to fund on average daily attendance, and only a few states are bold enough to still do it,” he said. “There’s no excuse for doing it. There’s no legitimate incentive that funding on average daily attendance will, you know, cause attendance to improve.”

Anglum and Shuls agreed that using average daily attendance penalizes poorer schools, although Shuls said there are pros and cons in all methods of calculating attendance. 

Another quirk of Missouri’s system is that while it “weights” students who are typically more costly to educate, it only does so when the percentage of students in a specific category exceeds a specific threshold. 

For example, the threshold for students receiving free and reduced-price lunch — a common way to estimate numbers of low-income students — is a bit more than 30%. Schools that serve a higher percentage than that get extra funding. Meanwhile, a district serving 25% students in that category doesn’t receive more funding than a district serving 5%.

Shuls, Baker and Anglum all criticized the use of thresholds. Shuls suggested the state could even differentiate the amounts granted for special-needs students — who can have very different funding needs — to better create a system where money “follows the student.” 

How Missouri compares to Kansas

Baker said that over the past decades, Kansas has strengthened its school finance system while Missouri’s has weakened. 

Baker formerly taught at the University of Kansas and was involved in discussions surrounding school finance reform in both Missouri and Kansas. He recently published “School Finance and Education Equity: Lessons from Kansas,” which he said includes many comparisons with Missouri.  

“The costs to get to the same outcomes are a little lower in Kansas, but Kansas also much more robustly funds their system,” Baker said. 

In Kansas, 58% of districts are spending above the adequate level and achieving results above the national average, Baker said. In Missouri, only 43% of districts are doing the same. 

Meanwhile, about 13% of Kansas districts are spending below the targets and achieving below the national average. Nearly 30% of Missouri districts are in the same boat. 

“There’s much more inequality in Missouri; there’s far more kids in inadequately funded districts that then have inadequate outcomes to go along with that,” Baker said. “Kansas has just done much better in that regard, over time.”

The Beacon is working on a larger story about Kansas’ school finance formula.

A report from the Missouri Budget Project shows Missouri’s overall K-12 funding target for each student, adjusted for inflation, is less than the 2007 amount — by about $1,000. 

Gleason, the project spokesperson, said that while Kansans reacted to abrupt funding cuts several years ago and restored funding, Missourians haven’t been as aware of gradual funding cuts. 

“Missouri has been more like the frog in the frying pan, or boiling water … We just haven’t noticed because it’s happened so slowly over time.”

The post Missouri’s school funding system undermines its own goals for equity, experts say appeared first on The Beacon.

Problems Delay Launch of NASA's SLS Rocket - Again [Published articles]

With 8.8 million pounds of thrust, NASA's SLS would've been the most powerful rocket ever launched into space, notes the Orlando Sentinel. But instead on Saturday morning, "NASA scrubbed its second attempt to launch the Artemis I mission into lunar orbit..." reports CNET. "During a press conference later in the day, Jim Free, an associate administrator at NASA Headquarters, said we shouldn't expect to see a third attempt within this launch period, which culminates Tuesday." (Though the mission manager the next launch attempt could be as late as mid-October.) "This time, the culprit was a liquid hydrogen leak that showed up while the team was loading the rocket's core stage...." According to the space agency, the leak occurred "while loading the propellant into the core stage of the Space Launch System rocket" and that "multiple troubleshooting efforts to address the area of the leak, by reseating a seal in the quick disconnect where liquid hydrogen is fed into the rocket, did not fix the issue." This is the second time the Artemis I mission has been delayed. Liftoff attempt No. 1 was scheduled for Monday, but launch director Charlie Blackwell-Thompson had to call a scrub then as well, because of an unyielding problem with what's known as an engine bleed test. (This process is meant to allow the engines to chill to the right temperature by releasing a small amount of the fuel). "We were unable to get the engines within the thermal conditions required to commit to launch," Artemis mission manager Mike Sarafin said during a press conference on Tuesday. "In combination with that, we also had a bent valve issue on the core stage, and it was at that point that the team decided to knock off the launch attempt for that day."

Read more of this story at Slashdot.

Space Weather Prediction Center [Published articles]

Peter Eckersley, co-creator of Let’s Encrypt, dies at just 43 [Published articles]

This site, like millions of others, has a certificate from Let's Encrypt. Farewell, Peter Eckersley, PhD, who helped make it all possible.

Visual Studio Code is designed to fracture | Geoffrey Huntley [Published articles]

You Have No Idea How Bad Europe’s Energy Crisis Is [Published articles]

“Be like the rock where the waves come and crash in succession: solid, motionless, watching the boiling waters calm down around you.” [Published articles]

For farmer Dave Brandt, being a meme is honest work [Published articles]


Dave Brandt doesn’t particularly like memes.

“The only one I’ve ever looked at was mine,” he told me over the phone in between tasks around his 1200-acre farm in central Ohio (that day, he had to fix a combine, mail cornmeal to farm-to-table customers, and prepare for a public speaking engagement).

To non-farmers, Brandt is simply another character in the internet’s cast of memes: a friendly, flannel-shirt-clad portrait of a stereotypical farmer. His tagline? “It ain’t much, but it’s honest work.”

But while Brandt’s face is well known by young people online, he’s also a bona fide celebrity IRL among regenerative farmers as a “cover crop guru.” Dave Brandt’s farm in Carroll, Ohio was a birthplace for soil health principles, and his now-iconic photo was taken in 2015 when the United States Department of Agriculture wrote about Brandt’s farming tactics in an article called "Soil Health Campaign Turns Two: Seeks to Unlock Benefits on- and off-the-Farm.” It wasn’t until three years later that he unwittingly became a meme.

Dave Brandt’s image took off in a 2018 Reddit post captioned, “When your teacher asks you why you have submitted only one paper of 20 paper homework.” Hey, it ain’t much, but it’s honest work!

Then, he started getting recognized on the street. It first happened in January 2020, when he was visiting Illinois. “A couple was walking down the street and asked me for a picture,” Brandt says. “I thought, ‘okay, I don’t care, sure.’”

Meme vs. reality

Unlike the guy whose face is used as ‘the worst person you know,’ Brandt thinks it’s fun to be a meme. After all, “it ain’t much but it’s honest work” is a phrase he really does say all the time, at least according to his grandson.

When Brandt, now 75, is not working in the dirt or talking about dirt, he enjoys reading books with titles like Dirt and Dirt to Soil, and offering affectionate advice (mostly about agriculture). “I’ve probably made more mistakes than most of ya,” he once said to an audience of enraptured farmers in a presentation on his soil journey. On top of maintaining his Ohio farm, he spends about a third of his time on speaking engagements, going all around the world. In 2020, as his face was illuminated on smartphones, he was speaking at a NATO conference about soil and climate change.

He grew up at his grandfather’s farm and admits he was a “farm boy not very interested in school.” He was, he says, “more interested in daydreaming about tractors I could drive when I could get home.” He got married to his high school sweetheart, Kendra, and then two weeks later was drafted into the Vietnam War. Brandt spent two years in the Marine Corps working with armored vehicles at the demilitarized zone between North and South Vietnam. His experience with tractors “most definitely” helped him out. But when he returned to Ohio, tragedy struck. His father died in a tractor accident and Dave was forced to sell his farm and much of his equipment. When they started over the next year, Dave and Kendra didn’t have any tillage equipment to break up the soil.

His field

In 1978, Dave Brandt decided to plant a cereal rye cover crop to deal with the erosion on his hilly clay soils with poor drainage, and he’s never looked back. He calls them the “anchor of a diverse crop rotation” which makes his offseason fields look like meadows instead of barren plots of land.

His crops aren’t particularly unique (mostly corn, soybeans, and wheat) but his novel cover cropping tactics make his practice unconventional. In the mid-’90s, his farm started using several mixes of cover crops whose roots could break up the soil better than tilling while regenerating the chemicals, such as nitrogen, that are paramount to healthy soil. It was a breakthrough: less input was getting him more output, and his soil was as rich and wormy as ever. In his three-and-a-half decades of farming, he’s drastically decreased his use of fertilizers, fungicide, herbicide, and insecticide, and his land blooms in the offseason with all sorts of vegetation — sunflowers, radishes, various grasses, and more.

It’s a family affair. “My son is a polymer chemist, and he works eight hours and 10 hours a day, then maybe works one hour or two in the evening on a farm,” Brandt said. His wife, Kendra, worked on the farm, too, before she died last year following a seven-year fight with cancer. His grandson works with him full-time.

The Brandt family operates Walnut Creek Seeds to provide education and materials to other farmers (including backyard gardeners) on cover cropping, and aims to prescribe farmers the best cover crops for their land. He hopes, if anything, that his digital stardom gets more people interested in regenerative farming. “Dupont and Bayer don't care about you or your soil or our wildlife,” he says.

Conversely, Brandt compares the plants of a cover crop blend to a family. “We’d like to have what I call a community working together,” he says. “We have all the plants doing their thing.”

Nichelle Nichols Will Rest Among the Stars [Published articles]

Screenshot: NASA

On board a rocket named the Vulcan, a symbolic portion of Star Trek legend Nichelle Nichols’ ashes (donated by her son, Kyle Johnston) will be sent into space as a part of a deep-space trip that will honor many Star Trek luminaries. Gene Roddenberry and his wife, Majel Barrett Roddenberry, James “Scotty” Doohan, and VFX pioneer Douglass Trumbull will also be represented in the Celestis flight.

There is also a memorial available on the Celestis website for any fans who wish to celebrate Nichols’ life and pay tribute to the Star Trek star, as well as an opportunity to send in writing, music, photos, scripts, and any other kind of media. (Fanfic writers, I’m looking at you to represent!) According to the press release, “all names and messages will be digitized and launched with her on her journey” using the “Celestis Mindfile,” which is a suitably Star Trek name.

Nichols famously played Nyota Uhura on Star Trek, and was an instrumental force for good on television and in the real world. An activist as well as an icon, she worked with NASA to promote more diversity within STEM fields and used her position to promote representation and civil right across the board. Nichols passed away on July 30, 2022.


Want more io9 news? Check out when to expect the latest Marvel and Star Wars releases, what’s next for the DC Universe on film and TV, and everything you need to know about House of the Dragon and Lord of the Rings: The Rings of Power.

PostgreSQL 14 Internals book – Part II [Published articles]

FreeBSD on the Framework Laptop [Published articles]

A (a few) ops lessons we all learn the hard way [Published articles]

ops is hard. what have we learned so far?

Internet of Things [Published articles]

Managed expectations manage suffering [Published articles]

Rituals for Engineering Teams [Published articles]

Last weekend I happened to pick up a book called “Rituals For Work: 50 Ways To Create Engagement, Shared Purpose, And A Culture That Can Adapt To Change.” It’s a super quick read, more comic book than textbook, but I liked it.

It got me thinking about the many rituals I have initiated and/or participated in over the course of my career. Of course, I never thought of them as such — I thought of them as “having fun at work” 🙃 — but now I realize these rituals disproportionately contribute to my favorite moments and the most precious memories of my career.

Rituals (a definition): Actions that a person or group does repeatedly, following a similar pattern or script, in which they’ve imbued symbolism and meaning.

I think it is extremely worth reading the first 27 pages of the book — the Introduction and Part One. To briefly sum up the first couple chapters: the power of creative rituals comes from their ability to link the physical with the psychological and emotional, all with the benefit of “regulation” and intentionality. Physically going through the process of a ritual helps people feel satisfied and in control, with better emotional regulation and the ability to act in a steadier and more focused way. Rituals also powerfully increase people’s sense of belonging, giving them a stable feeling of social connection. (p. 5-6)

The thing that grabbed me here is that rituals create a sense of belonging. You show that you belong to the group by participating in the ritual. You feel like you belong to the group by participating in the ritual. This is powerful shit!

It seems especially relevant these days when so many of us are atomized and physically separated from our teammates. That ineffable sense of belonging can make all the difference between a job that you do and a role that feeds your soul. Rituals are a way to create that sense of belonging. Hot damn.

So I thought I’d write up some of the rituals for engineering teams I remember from jobs past. I would love to hear about your favorite rituals, or your experience with them (good or bad). Tell me your stories at @mipsytipsy. 🙃

Rituals at Linden Lab

Feature Fish Freeze

At Linden Lab, in the ancient era of SVN, we had something called the “Feature Fish”. It was a rubber fish that we kept in the freezer, frozen in a block of ice. We would periodically cut a branch for testing and deployment and call a feature freeze. Merging code into the branch was painful and time consuming, so If you wanted to get a feature in after the code freeze, you had to first take the fish out of the freezer and unfreeze it.

This took a while, so you would have to sit there and consider your sins as it slowly thawed. Subtext: Do you really need to break code freeze?

Stuffy the Code Reviewer

You were supposed to pair with another engineer for code review. In your commit message, you had to include the name of your reviewer or your merge would be rejected. But the template would also accept the name “Stuffy”, to confess that your only reviewer had been…Stuffy, the stuffed animal.

However if your review partner was Stuffy, you would have to narrate the full explanation of Stuffy’s code review (i.e., what questions Stuffy asked, what changes he suggested and what he thought of your code) at the next engineering meeting. Out loud.

Shrek Ears

We had a matted green felt headband with ogre ears on it, called the Shrek Ears. The first time an engineer broke production, they would put on the Ears for a day. This might sound unpleasant, like a dunce cap, but no — it was a rite of passage. It was a badge of honor! Everyone breaks production eventually, if they’re working on something meaningful.

If you were wearing the Shrek Ears, people would stop you throughout the day and excitedly ask what happened, and reminisce about the first time they broke production. It became a way for 1) new engineers to meet lots of their teammates, 2) to socialize lots of production wisdom and risk factors, and 3) to normalize the fact that yes, things break sometimes, and it’s okay — nobody is going to yell at you. ☺

This is probably the number one ritual that everybody remembers about Linden Lab. “Congratulations on breaking production — you’re really one of us now!”

Vorpal Bunny

vorpal bunny

We had a stuffed Vorpal Bunny, duct taped to a 3″ high speaker stand, and the operations engineer on call would put the bunny on their desk so people knew who it was safe to interrupt with questions or problems.

At some point we lost the bunny (and added more offices), but it lingered on in company lore since the engineers kept on changing their IRC nick to “$name-bunny” when they went on call.

There was also a monstrous, 4-foot-long stuffed rainbow trout that was the source of endless IRC bot humor… I am just now noticing what a large number of Linden memories involve stuffed animals. Perhaps not surprising, given how many furries were on our platform ☺

Rituals at Parse

The Tiara of Technical Debt

Whenever an engineer really took one for the team and dove headfirst into a spaghetti mess of tech debt, we would award them the “Tiara of Technical Debt” at the weekly all hands. (It was a very sparkly rhinestone wedding tiara, and every engineer looked simply gorgeous in it.)

Examples included refactoring our golang rewrite code to support injection, converting our entire jenkins fleet from AWS instances to containers, and writing a new log parser for the gnarliest logs anyone had ever seen (for the MongoDB pluggable storage engine update).

Bonfire of the Unicorns

We spent nearly 2.5 years rewriting our entire ruby/rails API codebase to golang. Then there was an extremely long tail of getting rid of everything that used the ruby unicorn HTTP server, endpoint by endpoint, site by site, service by service.

When we finally spun down the last unicorn workers, I brought in a bunch of rainbow unicorn paper sculptures and a jug of lighter fluid, and we ceremonially set fire to them in the Facebook courtyard, while many of the engineers in attendance gave their own (short but profane) eulogies.

Mission Accomplished

This one requires a bit of backstory.

For two solid years after the acquisiton, Facebook leadership kept pressuring us to move off of AWS and on to FB infra. We kept saying “no, this is a bad idea; you have a flat network, and we allow developers all over the world to upload and execute random snippets of javascript,” and “no, this isn’t cost effective, because we run large multi-terabyte MongoDB replica sets by RAIDing together multiple EBS volumes, and you only have 2.5TB FusionIO (for extremely high-perf mysql/RocksDB) and 40 TB spinning rust volumes (for Hadoop), and also it’s impossible to shrink or slice up replsets”, and so forth. But they were adamant. “You don’t understand. We’re Facebook. We can do anything.” (Literal quote)

Finally we caved and got on board. We were excited! I announced the migration and started providing biweekly updates to the infra leadership groups. Four months later, when the  migration was half done, I get a ping from the same exact members of Facebook leadership:

“What are you doing?!?”
“Migrating!”
“You can’t do that, there are security issues!”
“No it’s fine, we have a fix for it.”
“There are hardware issues!”
“No it’s cool, we got it.”
You can’t do this!!!”

ANYWAY. To make an EXTREMELY long and infuriating story short, they pulled the plug and canned the whole project. So I printed up a ten foot long “Mission Accomplished” banner (courtesy of George W Bush on the aircraft carrier), used Zuck’s credit card to buy $800 of top-shelf whiskey delivered straight to my desk (and cupcakes), and we threw an angry, ranty party until we all got it out of our systems.

Blue Hair

I honestly don’t remember what this one was about, but I have extensive photographic evidence to prove that I shaved the heads of and/or dyed the hair blue of at least seven members of engineering. I wish I could remember why! but all I remember is that it was fucking hilarious.

In Conclusion

Coincidentally (or not), I have no memories of participating in any rituals at the jobs I didn’t like, only the jobs I loved. Huh.

One thing that stands out in my mind is that all the fun rituals tend to come bottoms-up. A ritual that comes from your VP can run the risk of feeling like forced fun, in a way it doesn’t if it’s coming from your peer or even your manager. I actually had the MOST fun with this shit as a line manager, because 1) I had budget and 2) it was my job to care about teaminess.

There are other rituals that it does make sense for executives to create, but they are less about hilarious fun and more about reinforcing values. Like Amazon’s infamous door desks are basically just a ritual to remind people to be frugal.

Rituals tend to accrue mutations and layers of meaning as time goes on. Great rituals often make no sense to anybody who isn’t in the know — that’s part of the magic of belonging. 🥰

Now, go tell me about yours!

charity

How Thinking Hard Makes the Brain Tired [Published articles]

An anonymous reader shares a report: Physical labour is exhausting. A long run or a hard day's sweat depletes the body's energy stores, resulting in a sense of fatigue. Mental labour can also be exhausting. Even resisting that last glistening chocolate-chip cookie after a long day at a consuming desk job is difficult. Cognitive control, the umbrella term encompassing mental exertion, self-control and willpower, also fades with effort. But unlike the mechanism of physical fatigue, the cause of cognitive fatigue has been poorly understood. Previous accounts were incomplete. One of the most widely known, the biological one, draws from what is known about muscular fatigue. It posits that exerting cognitive control uses up energy in the form of glucose. At the end of a day spent intensely cogitating, the brain is metaphorically running on fumes. The problem with this version of events is that the energy cost associated with thinking is minimal. One analysis of previous studies suggests that cognitively overworked and "depleted" brains use less than one-tenth of a Tic-Tac's worth of additional glucose. If cognitive fatigue is not caused by a lack of energy, then what explains it? A team of scientists led by Antonius Wiehler of Pitie-Salpetriere University Hospital, in Paris, looked at things from what is termed a neurometabolic point of view. They hypothesise that cognitive fatigue results from an accumulation of a certain chemical in the region of the brain underpinning control. That substance, glutamate, is an excitatory neurotransmitter that abounds in the central nervous systems of mammals and plays a role in a multitude of activities, such as learning, memory and the sleep-wake cycle. In other words, cognitive work results in chemical changes in the brain, which present behaviourally as fatigue. This, therefore, is a signal to stop working in order to restore balance to the brain. In their new paper in Current Biology, the researchers describe an experiment they undertook to explain how all this happens.

Read more of this story at Slashdot.

‘Ring Nation’ is what happens when America's Funniest Home Videos meets Black Mirror [Published articles]

Wanda Sykes has signed on to host an America's Funniest Home Videos type of TV show. It won't be showing clips deliberately captured using cameras or phones, though — nope, it will feature videos captured by Ring doorbells and smart home cameras. The show is called Ring Nation, and it's a production by MGM Television and Big Fish Entertainment. If you're wondering what the common denominator is between the three, it's none other than Amazon. The e-commerce giant owns MGM and Big Fish, and it purchased Ring's smart doorbell business for $1 billion in 2018.

According to Deadline, Ring Nation will showcase viral videos that feature content such as neighbors saving neighbors, marriage proposals, military reunions and animals doing silly things. In other words, videos you'll probably come across online if you frequent social networks, unless the show will also feature fresh content that could potentially go viral as shared by Ring owners. 

Barry Poznick, president of alternative television & Orion TV at MGM, said: "From the incredible, to the hilarious and uplifting must-see viral moments from around the country every day, Ring Nation offers something for everyone watching at home."

That Amazon wants to make videos captured by its smart doorbells a source of funny family TV can feel a bit too Black Mirror-esque, especially when you consider Ring's relationship with law enforcement. Senator Edward Markey recently shared a disclosure revealing that Amazon had provided Ring footage to law enforcement in the US eleven times without a court order or the user's consent. 

A company spokesperson justified Ring's actions and told us that the law authorizes companies "to provide information to government entities if the company believes that an emergency involving danger of death or serious physical injury to any person...requires disclosure without delay." Tweeting about his revelation, Markey said: "We cannot accept this surveillance as inevitable." He also used the disclosure as an example of why lawmakers should pass the Facial Recognition and Biometric Technology Moratorium Act, which he introduced in hopes of banning law enforcement's use of the technologies. 

Researchers Find Vulnerability In Software Underlying Discord, Microsoft Teams, and Other Apps [Published articles]

An anonymous reader quotes a report from Motherboard: A group of security researchers found a series of vulnerabilities in the software underlying popular apps like Discord, Microsoft Teams, Spotify and many others, which are used by tens of millions of people all over the world. At the Black Hat cybersecurity conference in Las Vegas on Thursday, the researchers presented their findings, detailing how they could have hacked people who use Discord, Microsoft Teams, and the chat app Element by exploiting the software underlying all of them: Electron, which is a framework built on the open source Chromium and the cross-platform javascript environment Node JS. In all these cases, the researchers submitted vulnerabilities to Electron to get them fixed, which earned them more than $10,000 in rewards. The bugs were fixed before the researchers published their research. Aaditya Purani, one of the researchers who found these vulnerabilities, said that "regular users should know that the Electron apps are not the same as their day-to-day browsers," meaning they are potentially more vulnerable. In the case of Discord, the bug Purani and his colleagues found only required them to send a malicious link to a video. With Microsoft Teams, the bug they found could be exploited by inviting a victim to a meeting. In both cases, if the targets clicked on these links, hackers would have been able to take control of their computers, Purani explained in the talk. For him, one of the main takeaways of their research is that Electron is risky precisely because users are very likely to click on links shared in Discord or Microsoft Teams.

Read more of this story at Slashdot.

CEO's LinkedIn Crying Selfie About Layoffs Met with Backlash [Published articles]

Ignore the haters, and other lessons learned from creating JSON5 [Published articles]

How Mars rovers have evolved in 25 years of exploring the Red Planet [Published articles]

Few things are harder than hurling a robot into space — and sticking the landing. On the morning of July 4, 1997, mission controllers at the Jet Propulsion Laboratory in Pasadena, Calif., were hoping to beat the odds and land a spacecraft successfully on the Red Planet.

Twenty-five years ago that little robot, a six-wheeled rover named Sojourner, made it — becoming the first in a string of rovers built and operated by NASA to explore Mars. Four more NASA rovers, each more capable and complex than the last, have surveyed the Red Planet. The one named Curiosity marked its 10th year of cruising around on August 5. Another, named Perseverance, is busy collecting rocks that future robots are supposed to retrieve and bring back to Earth. China recently got into the Mars exploring game, landing its own rover, Zhurong, last year.

Other Mars spacecraft have done amazing science from a standstill, such as the twin Viking landers in the 1970s that were the first to photograph the Martian surface up close and the InSight probe that has been listening for Marsquakes shaking the planet’s innards (SN Online: 2/24/20). But the ability to rove turns a robot into an interplanetary field geologist, able to explore the landscape and piece together clues to its history. Mobility, says Kirsten Siebach, a planetary scientist at Rice University in Houston, “makes it a journey of discovery.”

Red planet ramblings

Five U.S. rovers and one Chinese rover have reached Mars, all visiting different locations on the planet. Many focused on areas that may have once been wet and favorable for potential life.

Map of rover landing sites on Mars
NASA MARS TREK; E. OTWELLNASA MARS TREK; E. OTWELL

Each of the Mars rovers has gone to a different place on the planet, enabling scientists to build a broad understanding of how Mars evolved over time. The rovers revealed that Mars contained water, and other life-friendly conditions, for much of its history. That work set the stage for Perseverance’s ongoing hunt for signs of ancient life on Mars.

Off-road trips

The ruggedness of the rovers is a big factor in how far they travel and how long they operate. Three of the machines are still exploring.

Distance traveled by rovers over time on Mars
graph showing distance traveled by various rovers on Mars compared to Martian days active; Opportunity traveled the furthest and spent the most time on Mars
E. OtwellE. Otwell

Each rover is also a reflection of the humans who designed and built and drove it. Perseverance carries on one of its wheels a symbol of Mars rover tracks twisted into the double helix shape of DNA. That’s “to remind us, whatever this rover is, it’s of human origin,” says Jennifer Trosper, an engineer at the Jet Propulsion Lab, or JPL, who has worked on all five NASA rovers. “It is us on Mars, and kind of our creation.”

Sojourner

illustration of the Sojourner rover
Cornelia Li

SIZE: Microwave oven

FUN FACTS:

  • Had to stay close to its lander to communicate with Earth
  • Analyzed 15 Martian rocks

The little microwave that could

Sojourner, that first rover, was born in an era when engineers weren’t sure if they even could get a robot to work on Mars. In the early 1990s, then-NASA Administrator Daniel Goldin was pushing the agency to do things “faster, better and cheaper” — a catchphrase that engineers would mock by saying only two of those three things were possible at the same time. NASA had no experience with inter­planetary rovers. Only the Soviet Union had operated rovers — on the moon in 1970 and 1973.

JPL began developing a Mars rover anyway. Named after the abolitionist Sojourner Truth, the basic machine was the size of a microwave oven. Engineers were limited in where they could send it; they needed a large flat region on Mars because handling a precision landing near mountains or canyons was beyond their abilities. NASA chose Ares Vallis, a broad outflow channel from an ancient flood, and the mission landed there successfully.

Sojourner spent nearly three months poking around the landscape. It was slow going. Mission controllers had to communicate with Sojourner constantly, telling it where to roll and then assessing whether it had gotten there safely. They made mistakes: One time they uploaded a sequence of computer commands that mistakenly told the rover to shut itself down. They recovered from that stumble and many others, learning to quickly fix problems and move forward.

photo of the Sojourner rover with its landing ramp in the foreground
In 1997, NASA’s first rover, Sojourner, rolled down a landing ramp and became the first mobile Mars robot. Solar panels provided power throughout its 12-week mission.JPL-CALTECH/NASA

Although Sojourner was a test mission to show that a rover could work, it managed to do some science with its one X-ray spectrometer. The little machine analyzed the chemical makeup of 15 Martian rocks and tested the friction of the Martian soil.

After surviving 11 weeks beyond its planned one-week lifetime, Sojourner ultimately grew too cold to operate. Trosper was in mission control when the rover died on September 27, 1997. “You build these things, and even if they’re well beyond their lifetime, you just can’t let go very easily, because they’re part of you,” she says.

image of Jennifer Trosper standing in front of a model of the Perseverance rover
Jennifer Trosper, an engineer at the Jet Propulsion Laboratory, is part of a small group of people who have worked on all five NASA Mars rovers. Here she is in 2021 with a model of Perseverance.CHRISTOPHER MICHEL/WIKIMEDIA COMMONS (CC BY-SA 4.0)

Spirit & Opportunity

illustration of the Spirit and Opportunity rovers
Cornelia Li

SIZE: Golf cart

FUN FACTS:

  • Spirit spotted minerals possibly left by an ancient hot spring
  • Opportunity found the first definitive evidence of ancient water on Mars
  • Spirit perished when it got stuck in a sand pit
  • Opportunity died in a dust storm, which cut  off solar power

Twin explorers

In 1998 and 1999, NASA hurled a pair of spacecraft at Mars; one was supposed to orbit the planet and another was supposed to land near one of the poles. Both failed. Stung from the disappointment, NASA decided to build a rover plus a backup for its next attempt.

Thus were born the twins Spirit and Opportunity. Each the size of a golf cart, they were a major step up from Sojourner. Each had a robotic arm, a crucial development in rover evolution that enabled the machines to do increasingly sophisticated science. The two had beefed-up cameras, three spectrometers and a tool that could grind into rocks to reveal the texture beneath the surface.

But there were a lot of bugs to work out. Spirit and Opportunity launched several weeks apart in 2003. Spirit got to Mars first, and on its 18th Martian day on the surface it froze up and started sending error messages. It took mission controllers days to sort out the problem — an overloaded flash-memory system — all while Opportunity was barreling toward Mars. Ultimately, engineers fixed the problem, and Opportunity landed safely on the opposite side of the planet from Spirit.

Both rovers lasted years beyond their expected three-month lifetimes. And both did far more Martian science than anticipated.

Spirit broke one of its wheels early on and had to drive backward, dragging the broken wheel behind it. But the rover found plenty to do near its landing site of Gusev crater, home to a classic Mars landscape of dust, rock and hills. Spirit found rocks that appeared to have been altered by water long ago and later spotted a pair of iron-rich meteorites. The rover ultimately perished in 2010, stuck in a sand-filled pit. Mission controllers tried to extract it in an effort dubbed “Free Spirit,” but salts had precipitated around the sand grains, making them particularly slippery.

Opportunity, in contrast, became the Energizer Bunny of rovers, exploring constantly and refusing to die. Immediately after landing in Meridiani Planum, Opportunity had scientists abuzz.

pale rock on the Martian surface with Opportunity's arm in the foreground
The pale rock at center, seen beneath the Opportunity rover’s robotic arm in 2013, was one of many at the rover’s landing site that held long-awaited evidence that liquid water once flowed on Mars. JPL-CALTECH/NASA, CORNELL UNIV., ARIZONA STATE UNIV.

“The images that the rover first sent back were just so different from any other images we’d seen of the Martian surface,” says Abigail Fraeman, a planetary scientist at JPL. “Instead of these really dusty volcanic plains, there was just this dark sand and this really bright bedrock. And that was just so captivating and inspiring.”

Right at its landing site, Opportunity spotted the first definitive evidence of past liquid water on Mars, a much-anticipated and huge discovery (SN: 3/27/04, p. 195). The rover went on to find evidence of liquid water at different times in the Martian past. After years of driving, the rover reached a crater called Endeavour and “stepped into a totally new world,” Fraeman says. The rocks at Endeavour were hundreds of millions of years older than others studied on Mars. They contained evidence of different types of ancient water chemistry.

Opportunity ultimately drove farther than any rover on any extraterrestrial world, breaking a Soviet rover’s lunar record. In 2015, Opportunity passed 26.2 miles (42.2 km) on its odometer; mission controllers celebrated by putting a marathon medal onto a mock-up of the rover and driving it through a finish line ribbon at JPL. Opportunity finally died in 2019 after an intense dust storm obscured the sun, cutting off solar power, a must-have for the rover to recharge its batteries (SN: 3/16/19, p. 7).

The twin rovers were a huge advance over Sojourner. But the next rover was an entirely different beast.

Ashwin Vasavada stands amid models of various NASA Mars rovers
Mission project scientist Ashwin Vasavada stands with several rovers, which learn to traverse various surfaces in the Mars Yard at NASA’s Jet Propulsion Laboratory in Pasadena, Calif.JPL-Caltech/NASA

Curiosity

illustration of the Curiosity rover
Cornelia Li

SIZE: SUV

FUN FACTS:

  • Needed a sky crane with rockets to hover above ground and slowly lower the behemoth to the surface
  • Found evidence of an ancient lake, sniffed methane gas, measured radiation levels

The SUV of rovers

By the mid-2000s, NASA had decided it needed to go big on Mars, with a megarover the size of a sports utility vehicle. The one-ton Curiosity was so heavy that its engineers had to come up with an entirely new way to land on Mars. The “sky crane” system used retro-rockets to hover above the Martian surface and slowly lower the rover to the ground.

Against all odds, in August 2012, Curiosity landed safely near Mount Sharp, a 5-kilometer-high pile of sediment within the 154-kilometer-wide Gale crater (SN: 8/25/12, p. 5). Unlike the first three Mars rovers, which were solar-powered, Curiosity runs on energy produced by the radioactive decay of plutonium. That allows the rover to travel farther and faster, and to power a suite of sophisticated science instruments, including two chemical laboratories.

Curiosity introduced a new way of exploring Mars. When the rover arrives in a new area, it looks around with its cameras, then zaps interesting rocks with its laser to identify which ones are worth a closer look. Once up close, the rover stretches out its robotic arm and does science, including drilling into rocks to see what they are made of.

When Curiosity arrived near the base of Mount Sharp, it immediately spotted rounded pebbles shaped by a once-flowing river, the first close­up look at an ancient river on Mars. Then mission controllers sent the rover rolling away from the mountain, toward an area in the crater known as Yellowknife Bay. There Curiosity discovered evidence of an ancient lake that created life-friendly conditions for potentially many thousands of years.

Curiosity then headed back toward the foothills of Mount Sharp. Along the way, the rover discovered a range of organic molecules in many different rocks, hinting at environments that had been habitable for millions to tens of millions of years. It sniffed methane gas sporadically wafting within Gale crater, a still-unexplained mystery that could result from geologic reactions, though methane on Earth can be formed by living organisms (SN: 7/7/18, p. 8). The rover measured radiation levels across the surface — helpful for future astronauts who’ll need to gauge their exposure — and observed dust devils, clouds and eclipses in the Martian atmosphere and night sky.

photo of clouds above Gale crater on Mars
Shimmering clouds of ice crystals appear in the sky above Gale crater on Mars, as seen by the Curiosity rover in March 2021. The ability to drive across Mars gives rovers a humanlike ability to interact with the landscape. MSSS, JPL-Caltech/NASA

“We’ve encountered so many unexpectedly rich things,” says Ashwin Vasavada of JPL, the mission’s project scientist. “I’m just glad a place like this existed.”

Ten years into its mission, Curiosity still trundles on, making new discoveries as it climbs the foothills of Mount Sharp. It recently departed a clay-rich environment and is now entering one that is heavier in sulfates, a transition that may reflect a major shift in the Martian climate billions of years ago.

In the course of driving more than 28 kilometers, Curiosity has weathered major glitches, including one that shuttered its drilling system for over a year. And its wheels have been banged up more than earthbound tests had predicted. The rover will continue to roll until some unknown failure kills it or its plutonium power wanes, perhaps five years from now.

photo showing holes in two of Curiosity's wheels
Over nearly 10 years of driving on Mars’ rocky surface, Curiosity’s wheels have taken more of a beating than its designers expected. MSSS, JPL-Caltech/NASA

Perseverance

illustration of the Perseverance rover
Cornelia Li

SIZE: SUV

FUN FACTS:

  • Collecting rocks for later recovery mission
  • Traveled almost five kilometers in 30 Martian days
  • Helicopter Ingenuity was expected to fly five times, but it has made 29 trips so far

A rover and its sidekick

NASA’s first four rovers set the stage for the most capable and agile rover ever to visit Mars: Perseverance. Trosper likens the evolution of the machines to the growth of children. “We have a preschooler in Sojourner, and then … your happy-go-lucky teenagers in Spirit and Opportunity,” she says. “Curiosity is certainly a young adult that’s able to do a lot of things on her own, and Perseverance is kind of that high-powered mid­career [person] able to do pretty much anything you ask with really no questions.”

Perseverance is basically a copy of Curiosity built from its spare parts, but with one major modification: a system for drilling, collecting and storing slender cores of rock. Perseverance’s job is to collect samples of Martian rock for future missions to bring to Earth, in what would be the first robotic sample return from Mars. That would allow scientists to do sophisticated analyses of Martian rocks in their earthbound labs. “It feels, even more than previous missions, that we are doing this for the next generation,” Siebach says.

The rover is working fast. Compared with Curiosity’s leisurely exploration of Gale crater, Perseverance has been zooming around its landing site, the 45-kilometer-wide Jezero crater, since its February 2021 arrival. It has collected 10 rock cores and is already eyeing where to put them down on the surface for future missions to pick up. “We’re going to bring samples back from a diversity of locations,” says mission project scientist Kenneth Farley of Caltech. “And so we keep to a schedule.”

Perseverance went to Jezero to study an ancient river delta, which contains layers of sediment that may harbor evidence of ancient Martian life. But the rover slightly missed its target, landing on the other side of a set of impassable sand dunes. So it spent most of its first year exploring the crater floor, which turned out to be made of igneous rocks (SN: 9/11/21, p. 32). The rocks had cooled from molten magma and were not the sedimentary rocks that many had expected.

Scientists back on Earth will be able to precisely date the age of the igneous rocks, based on the radioactive decay of chemical elements within them, providing the first direct evidence for the age of rocks from a particular place on Mars.

image of a rock core sample collected by the Perseverance rover
Perseverance collected its 9th rock core, barely the size of a pinky finger, on July 7. Future missions will return the stored samples to Earth for study. JPL-CALTECH/NASA, ARIZONA STATE UNIV.

Once it finished exploring the crater floor in March, the rover drove quickly toward the delta. Each successive NASA rover has had greater skills in autonomous driving, able to identify hazards, steer around them and keep going without needing constant instructions from mission control.

Perseverance has a separate computer processor to run calculations for autonomous navigation, allowing it to move faster than Curiosity. (It took Curiosity two and a half years to travel 10 kilometers; Perseverance traveled that far in a little over a year.) “The rover drives pretty much every minute that we can give it,” Farley says.

In April, Perseverance set a Martian driving record, traveling nearly five kilometers in just 30 Martian days. If all goes well, it will make some trips up and down the delta, then travel to Jezero crater’s rim and out onto the ancient plains beyond.

Perseverance has a sidekick, Ingenuity, the first helicopter to visit another world. The nimble flier, only half a meter tall, succeeded beyond its designers’ wildest dreams. The helicopter made 29 flights in its first 16 months when it was only supposed to make five in one month. It has scouted paths ahead and scientific targets for the rover (SN Online: 4/19/22). Future rovers are almost certain to carry a little buddy like this.

image of an engineer working on the Perseverance rover in a clean room with light from above
An engineer at NASA’s Jet Propulsion Laboratory measures light on the Perseverance rover during a 2019 test. The rover landed on Mars last year and has been exploring it ever since.JPL-CALTECH/NASA

Zhurong

illustration of the Zhurong rover

Cornelia Li

SIZE: Golf cart

FUN FACTS:

  • Dormant now during Martian winter, but is set to reawaken in December
  • Found signs of recent water activity in a  geologically young setting
  • Found Martian soil similar to some of Earth’s desert sands

China’s debut

While the United States has led in Mars rover exploration, it is not the only player on the scene. In May 2021, China became the second nation to successfully place a rover on Mars. Its Zhurong rover, named after a mythological fire god, has been exploring part of a large basin in the planet’s northern hemisphere known as Utopia Planitia.

The landing site lies near a geologic boundary that may be an ancient Martian shoreline. Compared with the other Mars rover locations, Zhurong’s landing site is billions of years younger, “so we are investigating a different world on Mars,” says Lu Pan, a planetary scientist at the University of Copenhagen who has collaborated with Zhurong scientists.

In many ways, Zhurong resembles Spirit and Opportunity, in size as well as mobility. It carries cameras, a laser spectrometer for studying rocks and ground-penetrating radar to probe underground soil structures (SN Online: 5/19/21).

After landing, Zhurong snapped pictures of its rock-strewn surroundings and headed south to explore a variety of geologic terrains, including mysterious cones that could be mud volcanoes and ridges that look like windblown dunes. The rover’s initial findings include that the Martian soil at Utopia Planitia is similar to some desert sands on Earth and that water had been present there perhaps as recently as 700 million years ago.

In May, mission controllers switched Zhurong into dormant mode for the Martian winter and hope it wakes up at the end of the season, in December. It has already traveled nearly two kilometers across the surface, farther than the meager 100 meters that Sojourner managed. (To be fair, Sojourner had to keep circling its lander because it relied on that lander to communicate with Earth.)

selfie image taken by the Zhurong rover on the surface of Mars with its landing platform
The China National Space Administration released this image on June 11, 2021 of Zhurong with its landing platform on Mars.CNSA/Handout via Xinhua

From Sojourner to Zhurong, the Mars rovers show what humankind can accomplish on another planet. Future rovers might include the European Space Agency’s ExoMars, although its 2022 launch was postponed after Russia attacked Ukraine (SN: 3/26/22, p. 6). Europe terminated all research collaborations with Russia after the invasion, including launching ExoMars on a Russian rocket.

Vasavada remembers his sense of awe at the Curiosity launch in 2011: “Standing there in Florida, watching this rocket blasting off and feeling it in your chest and knowing that there’s this incredibly fragile complex machine hurtling on the end of this rocket.… It just gave me this full impression that here we are, humans, blasting these things off into space,” he says. “We’re little tiny human beings sending these things to another planet.”

Claydream Is an Inspiring Yet Cautionary Show-Biz Tale [Published articles]

Image: Oscilloscope Labs

Claydream, Marq Evans’ new documentary about animator Will Vinton, addresses the elephant in the room immediately: yes, this is the guy who lost his company to his most deep-pocketed investor, Nike founder Phil Knight. It’s something that looms over the film, but it’s not the only melancholy element that colors this portrait of Vinton’s life and career.

Made with the cooperation of Vinton himself, who died of cancer in 2018 but is interviewed extensively here, Claydream offers a visual history of his remarkable accomplishments. Not only do we get a look at the progression of Vinton’s work over the years (from Closed Mondays, the Oscar-winning 1974 short he created with Bob Gardiner, to his company’s instantly recognizable commercial work from the ‘80s and ‘90s, including the California Raisins), we also get access to home movies, as well as firsthand accounts from friends, family members, and former coworkers. After sparking to filmmaking while at UC Berkeley in the 1960s, Vinton (who prized experimentation and creative fulfillment above all else, and was definitely a bit of a hippie) set up a small workshop with his collaborators in Portland, Oregon, a location that kept their productions deliberately removed from the Hollywood machine—the same machine he’d end up pursuing years later, when Will Vinton Studios was at its peak.

Most of Claydream keeps the focus on Vinton’s work—again, this movie is a visual feast, jam-packed with clips and other ephemera (including answering-machine messages from a California Raisins-obsessed Michael Jackson) that illustrate the narrative of Vinton’s career every step of the way. But for all his success, and for the admirable way he bounced back from his periodic failures and missteps, he never achieved the heights of his idol, Walt Disney, whose life trajectory he emulated, down to plans for a never-realized “Claymation Station” amusement park. Though he was well-liked as a person, not everyone he worked with is full of praise; there were issues over the years of sharing credit with the other animators who toiled on his projects, as well as some bad business decisions that meant, for instance, that Will Vinton Studios didn’t share in the licensing for the insanely marketable California Raisins—and also that Vinton passed on selling his company to Pixar during its pre-Disney era. A contentious split with the troubled Gardiner soon after their shared Oscar win haunted Vinton until Gardiner’s death in 2005. But as Claydream amply illustrates, the Phil Knight debacle ended up being the biggest tragedy of Vinton’s creative life.

Neither Knight nor his son Travis Knight are interviewed in Claydream; we see them in deposition and archival footage only. Travis Knight, now a film director known for the stop-motion feature Kubo and the Two Strings as well as the live-action Transformers spin-off Bumblebee, comes off particularly badly just on the basis of the facts presented: a failed rapper, he was hired at Will Vinton Studios after his father invested in it, where he developed his (by all accounts) true talent and passion for animation. But there’s no escaping the “nepotism baby” aroma that envelops him in this context, especially when the documentary points out that he became head of Will Vinton Studios—renamed Laika—after Vinton, who was unable to rescue his financially struggling company, was pushed out.

It’s juicy show-biz stuff, for sure, but Vinton makes a point of turning what was obviously an incredibly devastating blow into something positive. Looking back several years after he lost his studio, he sounds genuinely proud of its continued success, specifically in the way that Laika—which has since become a Hollywood powerhouse with acclaimed titles like Coraline, ParaNorman, The Boxtrolls, Missing Link, and Knight’s Kubo—brought stop-motion to an ever-wider audience while innovating on the art form. It couldn’t have been easy for Vinton to make peace with the situation, but Claydream sure makes it seem like he was able to. Perhaps, as in his earliest days as a counterculture animator, it all came down to what really mattered: making an end product that was cool as it could possibly be. Even if Vinton wasn’t directly involved in any of Laika’s titles, his legacy lives on.

Claydream hits select theaters today, August 5.


Want more io9 news? Check out when to expect the latest Marvel and Star Wars releases, what’s next for the DC Universe on film and TV, and everything you need to know about House of the Dragon and Lord of the Rings: The Rings of Power.

Interview with the scientist who’s writing the Webb photos alt text [Published articles]

Anonymous Hacktivists Breach Russian Databases, Leak 'Massive' Amounts of Data [Published articles]

"The Anonymous declaration of cyberwar was a top news story despite no evidence," writes cybersecurity specialist Jeremiah Fowler (an American who worked in Kyiv for the last 10 years — until fleeing in February to Poland). To investigate, Fowler performed a random sampling of 100 exposed Russian databases — and discovered that 92 of them had indeed been compromised. "Anti-Russian hackers used a similar script to the infamous 'MeowBot' that changed the name of folders and deleted the contents of the files. " (For example, renaming the folders to "putin_stop_this_war".) And that was just the beginning, reports CNBC: Anonymous has claimed to have hacked over 2,500 Russian and Belarusian sites, said Fowler. In some instances, stolen data was leaked online, he said, in amounts so large it will take years to review. "The biggest development would be the overall massive number of records taken, encrypted or dumped online," said Fowler. Shmuel Gihon, a security researcher at the threat intelligence company Cyberint, agreed that amount of leaked data is "massive." "We currently don't even know what to do with all this information, because it's something that we haven't expected to have in such a short period of time," he said.... The more immediate outcome of the hacks, Fowler and Gihon agreed, is that Russia's cybersecurity defenses have been revealed as being far weaker than previously thought. Fowler's report argues that Anonymous has "rewritten the rules of how a crowdsourced modern cyberwar is conducted" — with the group also offering penetration testing to Ukraine, "finding vulnerabilities before Russia could exploit them." But in addition, Fowler writes, Anonymous's efforts have also "transformed into a larger operation that spread far beyond the Russian government, companies, or organizations, and included an information campaign aimed at Russian citizens." Some examples: Hacking Printers — Russian censorship has blocked many inside the country from knowing the true scale of the war and Russian losses. Anonymous hacked printers across Russia and printed uncensored facts or anti-propaganda and pro-ukrainian messages. The group claims to have printed over 100,000 documents. This also includes barcode printers at grocery stores where prices were changed and product names were changed to anti-war or pro-Ukrainian slogans.... RoboDial, SMS, and Email Spam — Almost everyone on earth has received some form of spam in the form of a phone call, text, or email message. These usually try to sell a service or scam victims out of money. Now this same technology has been used to bypass Russian censorship and inform citizens of news and messages they are forbidden to learn on state sponsored propaganda channels. Anonymous affiliated Squad303 claimed to have sent over 100 million messages to Russian devices.

Read more of this story at Slashdot.

Max Headroom Is Making a Comeback [Published articles]

He’s b-b-back! (Sorry, but there’s no avoiding it.) The ’80s pop culture icon Max Headroom is returning to a TV screen near you thanks to AMC Networks and Halt and Catch Fire creator co-Christopher Cantwell, who apparently isn’t done with ideas from the 1980s just yet.

Matt Frewer will return as the perpetually glitching AI TV personality for this new series, which is produced by Elijah Wood and Daniel Noah.

If you did not spend a lot of time in front of a television in the ’80s, Max Headroom was kind of confusing: Why is the robot head on the cover of Newsweek? But the satirical talk show host character was the star of a TV movie (1985’s Max Headroom: 20 Minutes into the Future), host of a music video show, the face of New Coke, and generally just an unavoidable part of that specific moment in time. No one has forgotten Max Headroom. As Variety notes, he’s referenced in everything from Agents of S.H.I.E.L.D. to a Selena Gomez video. (Editor’s note: To FARSCAPE, sorry, this is important to me.)

In the years since he first played Max Headroom, Matt Frewer has established himself as a powerhouse character actor; personally I can only see him as Orphan Black‘s creepy Dr. Leekie, but he’s been in the 1984 The Stand, Star Trek: The Next Generation, Falling Skies, The Librarians, Eureka, Fear the Walking Dead, and The Magicians, while also providing voices for a ton of animated shows.

No details or further casting have been announced for the new Max Headroom drama series.


Buy Face from:

After a minimum wage increase, workers become more productive. On the whole, it leads to welfare improvements for both employed and unemployed workers (i.e. the minimum wage increase is not counterproductive), but reduces company profits. [Data: 40,000 retail workers in large US stores] [Published articles]

A guide to stargazing and observing the cosmos from Kansas City [Published articles]

Get a close-up look at the celestial bodies in our universe from one of the Kansas City region's observatories, or see shooting stars with your naked eye during the Perseids meteor shower in August.

Amateur astronomers’ images of a rare double aurora may unlock its secrets [Published articles]

What happens when two different kinds of auroras get together? One spills the other’s secrets.

Amateur astronomers have captured a strange combination of red and green auroras on camera, and physicists have now used these images to learn what may trigger the more mysterious part of the lightshow.

Photographer Alan Dyer was in his backyard in Strathmore, Canada, when he saw the lights dancing overhead and started filming. “I knew I had something interesting,” says Dyer, who also writes about astronomy. What he didn’t know was that he had just made the most complete recording of this rarely seen phenomenon.

At a glance, Dyer’s video looks like a celestial watermelon. The rind, a rippling green aurora, is well understood: It appears when the solar wind energizes protons trapped within Earth’s magnetic field, which then rain down and knock electrons and atoms around (SN: 12/10/03). That action is what gives the phenomenon — called a proton aurora — its name.

The swath of fruity magenta is more mysterious: Though scientists have known about these “stable auroral red arcs” for decades, there’s no widely accepted proof of how they form. One popular theory is that part of Earth’s magnetic field can heat up the atmosphere and, like proton rain, jostle particles.   

Researchers had seen both of these particular kinds of aurora before, says Toshi Nishimura, a space physicist at Boston University, but the combination was mysterious. “Scientists didn’t have a good idea of why they could be together.”

Alan Dyer’s footage of this rare double aurora, a time lapse captured over 33 minutes on October 12, 2021, is helping physicists tease out clues to what causes the red glow.

Along with satellite observations, Dyer’s images and similar ones captured by other amateur astronomers in Canada and Finland show how the two phenomena may be related, Nishimura’s team reports in the July JGR Space Physics. Thin rays in the red aurora trace the paths of electrons as they fall along the Earth’s magnetic field. So just as proton rain triggers the green aurora, electron rain appears to trigger the red one, with the solar wind powering both at the same time. Since the electrons carry less energy than the protons, they make for a more reddish color. 

But electron rain might not be the only way to produce the stable auroral red arcs, cautions Brian Harding, a space physicist at the University of California, Berkeley. Either way, he says, the results are exciting because they show what’s going on is more complicated than researchers thought.

Those complications are important to understand. The auroras Dyer saw, though beautiful, are danger zones for radio communication and GPS systems (SN: 8/13/17). As Nishimura puts it: If you were driving under a subauroral red arc, your GPS might tell you to veer into a field.

Until scientists better understand these red glows, they won’t be able to forecast space weather like they do normal weather, Harding explains. “You want to make sure that you can predict stuff like this,” he says.

The new results would not have been possible without the citizen scientists who took the photos, Nishimura says. “This is a new way of doing research…. When they take more and more cool images, they find more and more things that we don’t know about.”

According to Dyer, more photos are exactly what’s coming. “We can make a unique contribution to science,” he says.  After all, “you never know what’s going to appear.”

More Relaxed [Published articles]

Dungeons & Dragons Trailer Rolls 20 For Monsters, Thieving, Fighting and Fun [Published articles]

The Dungeons & Dragons: Honor Among Thieves movie starring Chris Pine, Regé -Jean Page, Hugh Grant, Michelle Rodriguez, Sophia Lillis, and Justice Smith now has a trailer that looks… kinda fun?

As you can see above, Pine’s character seems to have accidentally unleashed a great evil upon the world (oops!). He’s ready to stop it, however (just after getting a drink), and has gathered a group of thieves with various skillsets to help him do so.

Here’s Paramount Pictures’ official synopsis:

A charming thief and a band of unlikely adventurers undertake an epic heist to retrieve a lost relic, but things go dangerously awry when they run afoul of the wrong people. Dungeons & Dragons: Honor Among Thieves brings the rich world and playful spirit of the legendary roleplaying game to the big screen in a hilarious and action-packed adventure.

There are also a bunch of monsters, animals, and creatures on display in the trailer. I admittedly am not well-versed in the numerous adversaries, foes, and friends from the D&D universe, but the dragon spewing purple mud(?) was rather interesting, and I think the owl-bear is my new favorite creature.

We’ll most likely see many more nods to the game when Dungeons & Dragons: Honor Among Thieves premieres in theaters on March 3, 2023.

Dogs are great sniffers. A newfound nose-to-brain connection helps explain why [Published articles]

A dog’s brain is wired for smell. Now, a new map shows just how extensive that wiring is.

Powerful nerve connections link the dog nose to wide swaths of the brain, researchers report July 11 in the Journal of Neuroscience. One of these canine connections, a hefty link between areas that handle smell and vision, hasn’t been seen before in any species, including humans.

The results offer a first-of-its-kind anatomical description of how dogs “see” the world with their noses. The new brain map is “awesome, foundational work,” says Eileen Jenkins, a retired army veterinarian and expert on working dogs. “To say that they have all these same connections that we have in humans, and then some more, it’s going to revolutionize how we understand cognition in dogs.”

In some ways, the results aren’t surprising, says Pip Johnson, a veterinary radiologist and neuroimaging expert at Cornell University College of Veterinary Medicine. Dogs are superb sniffers. Their noses hold between 200 million and 1 billion odor molecule sensors, compared with the 5 million receptors estimated to dwell in a human nose. And dogs’ olfactory bulbs can be up to 30 times larger than people’s. But Johnson wanted to know how smell information wafts to brain regions beyond the obvious sniffing equipment.

To build the map, Johnson and colleagues performed MRI scans on 20 mixed-breed dogs and three beagles. The subjects all had long noses and medium heads, and were all probably decent sniffers. Researchers then identified tracts of white matter fibers that carry signals between brain regions. A method called diffusion tensor imaging, which relies on the movement of water molecules along tissue, revealed the underlying tracts, which Johnson likens to the brain’s “road network.”

After odor information enters the nose, it whizzes to the olfactory bulb, a brain structure that sits behind the dogs’ eyes. But from there, it wasn’t clear where the signals went next.  When Johnson looked for the tracts in the dog MRI data, she was blown away. “I just kept finding these huge pathways,” she says. “They seem like information freeways running from the nose back into the brain.”

Sniff system

In the dog brain, sturdy neural tracts run from the olfactory bulb (bottom left) to five distinct spots. A newfound connection (orange) links the olfactory system to vision, something that has so far been found only in dogs. The others lead to the cortical spinal tract (turquoise path), which helps trigger instinctual behaviors; the piriform lobe (green path), which may play a role in odor perception; the limbic system (blue path), which is involved with behavior and emotion; and the entorhinal cortex (pink path), which helps handle memory.

diagram showing the inside of a dog brain
P. Johnson, generated using TrackVis software

This new dog brain map contains some familiar roads, including those that connect the olfactory bulb to brain areas associated with memories and emotions. In people, those roads explain why a whiff of perfume can transport a person back in time.

But one tract was totally new. This road, thick and obvious, connected the olfactory bulb to the occipital lobe, the part of the dog brain that handles vision. “There have been lots of people who theorized that this connection existed, based on the behavior of trained dogs and detection dogs,” says Jenkins, who currently practices at Huntsville Veterinary Specialists & Emergency in Alabama and who was not involved in this study. “But nobody has been able to prove it. This is fabulous.”

Dogs use all their senses to evaluate their environment. But this newfound connection between smell and sight suggests that the two are intricately linked. Perhaps this anatomical link could be why smell can often compensate when a dog’s sight goes, Johnson says. “Blind dogs can still play fetch.”

Breeding can affect the shapes of dog brains, neuroscientist Erin Hecht of Harvard University and colleagues have found (SN: 9/2/19). It would be interesting to see how these olfactory tracts look in different dog breeds, including scent hounds bred and trained for jobs such as hunting, finding disaster survivors or identifying diseases like cancer or COVID-19, Hecht says (SN: 6/1/22). “This study lays a foundation for future work,” she says.

Johnson and her colleagues aim to explore olfactory tracts of other animals. “I have actually had a play with some cat data,” she says. “Cats have the most amazing olfactory system too, and probably more connections than the dog that I can see.” But dog people, settle down. “That’s only preliminary data,” she quickly adds.

Lego Kansas City Skyline Instructions [Published articles]

I’m Brandon Sanderson, a bestselling fantasy author who somehow produced the highest-funded Kickstarter campaign of all time. AMA! [Published articles]

I’m Brandon Sanderson, a bestselling fantasy author. Best known for The Stormlight Archive, Mistborn, and for finishing Robert Jordan’s The Wheel of Time, I’m now also known for having the highest-funded campaign in Kickstarter’s history for four books I wrote during the quarantine. If you want to stay up to date with me, you should check out my YouTube channel (where you can watch me give my answers to the questions below) and my Facebook, Twitter, and Instagram. Ask me any questions you like, but I’m less likely to answer questions with massive spoilers for the books. I’ll be taking questions today only.

PROOF: https://i.redd.it/b41y1xaiq7a91.jpg

EDIT: I'm off the livestream and have had some dinner. The transcription of some questions is still coming, as...well, I talk a lot. Those answers will be posted soon, or you can see them on the VOD of my answers on the YouTube channel.

Apologies for the stream-of-consciousness wall-of-text answers. This was a new thing for us, finding a way for me to be able to give answers for people while also getting piles of pages signed. I hope you can make sense of the sometimes rambling answers I give. They might flow better if you watch them be spoken.

Thanks, all, for the wonderful AMA. And as I said, some answers are still coming (and I might pop in and write out a few others that I didn't get to.)

--Brandon

submitted by /u/mistborn to r/books
[link] [comments]

We’re heading for a messy, and expensive, breakup with natural gas [Published articles]

Russia’s invasion of Ukraine has exacerbated a number of fault lines already present within the global energy supply chain. This is especially true in Europe, where many countries were reliant on the superstate's natural resources, and are now hastily looking to cut ties before the supply is shut off. This has revealed the fragility of Europe’s energy market, and caused it to drive up demand and prices for consumers all over the globe.

In the UK, things are becoming increasingly dire and energy prices are skyrocketing. Bad planning on the infrastructure side and the cancellation of several major domestic energy efficiency programs are exacerbating the problem. It’s clear that real, useful action on the national level isn’t coming any time soon. So, I wondered, what would happen if I, personally, simply tried to break up with natural gas on my own? It’s relatively straightforward but, as it turns out, it comes at a cost that only one percenters will be able to bear. 

Dan Cooper: Energy consumer

I live in a four-bedroom, end-terraced house that’s around 150 years old and I’ve tried, as best as I can, to renovate it in an eco-friendly way. Since we bought it almost a decade ago, my wife and I have insulated most of the rooms, installed a new gas central heating system and hot water cylinder. We are, like nearly 20 million other households in the UK, reliant on natural gas to supply our home heating, hot water and cooking. And in the period between January 8th and April 7th, 2022, I was billed on the following usage:

Usage (kWh)

Cost Per Unit (GBP)

Cost (GBP)

Electricity (incl. standing charge)

861

0.32

£307.18

Gas (incl. standing charge)

8696.7

0.753

£678.80

Total (incl. tax and other charges)

£1,035.28

Essentially, I paid around $1,300 for my natural gas and electricity in the first quarter of 2022. That figure is likely to rise significantly, as the UK’s mandatory price cap on energy rose by more than 50 percent in April. A further price rise is scheduled for October, with the figure set at £2,800 per year, even though wholesale energy prices are no longer increasing. It’s likely that my energy bill for the first quarter of 2023 will be nearly twice what I’ve just paid. In 2020, the UK reported that 3.16 million households were unable to pay for their energy costs; that figure is likely to leap by 2023.

In the US, the EIA says that monthly utility bills rose to a national average of $122 in 2021, with Hawaii ($178 per month) and Utah ($82 per month) the most expensive and cheapest state to buy energy in. The average price per kWh is around 13.7 cents, which is less than half the comparable price in the UK as it currently stands. For natural gas, the average natural gas price for residential customers was $10.84 per thousand cubic feet in 2020.

The gas problem

MARSAXLOKK, MALTA APRIL 26: Photo shows a moored floating liquefied natural gas LNG storage unit, which provides LNG for the nearby Delimara power station in Marsaxlokk, Malta. (Photo by Chen Wenxian/Xinhua via Getty Images)
Xinhua News Agency via Getty Images

Much of Europe is reliant on natural gas, a significant proportion of which was supplied by Russia. Despite a rapid decline in domestic production, Europe sought to make natural gas the bedrock of its energy policy in the medium term. A 2013 policy paper written by Sami Andoura and Clémentine d’Oultremont outlined the reasons why officials were banking on it. “An economically attractive option for investors, a potential backup source for renewables and the cleanest fossil fuel, natural gas is expected to play an important role in the European transition towards a low-carbon economy by 2050.” This is despite the fact that “European energy resources are being depleted, and energy demand is growing.”

In 2007, then EU Energy Commissioner Andris Piebalgs said that the bloc is “dependent on imports for over one half of our energy use.” He added that energy security is a “European security issue,” and that the bloc was vulnerable to disruption. “In 10 years, from 1995 to 2005, natural gas consumption in the EU countries has increased from 369 billion to 510 billion m3 [of gas] year,” he said. He added that the EU’s own production capacity and reserves peaked in the year 2000.

The EU’s plan was to pivot toward Liquified Natural Gas (LNG), methane which has been filtered and cooled to a liquid for easier transportation. It enables energy supplies from further afield to be brought over to Europe to satisfy the continent’s need for natural gas. But the invasion of Ukraine by Russia has meant that this transition has now needed to be accelerated as leaders swear off Russian-sourced gas and oil. And while the plan is to push more investment into renewables, LNG imports are expected to fill much of the gap for now.

Except, and this is crucial, many of the policy decisions made during this period seem to be in the belief that nothing bad would, or could, disrupt supply. Here in the UK, wholesale gas prices have risen five times since the start of 2021 but there’s very little infrastructure available to mitigate price fluctuations. 

The Rough Field is a region in the North Sea situated 18 miles off the coast of Yorkshire, and was previously a source of natural gas for the UK. In 1985, however, it was converted into a natural gas storage facility with a capacity of 3.31 billion cubic meters. This one facility was able to fulfill the country’s energy needs for a little more than a week at a time and was considered a key asset to maintaining the UK’s energy security.

However, Centrica, the private company spun out of the former state-owned British Gas, opted to close the field in 2017. It cited safety fears and the high cost of repair as justification for the move, saying that alternative sources of gas – in the form of LNG – were available. At the time, one gas trader told Bloomberg that the closure would “boost winter prices” and “create seasonal swings in wholesale energy costs.” He added that the UK would now be “competing with Asia for winter gas cargoes,” raising prices and increasing reliance on these shipments. 

And, unsurprisingly, the ramifications of this decision were felt in the summer of 2017 when a pair of LNG tankers from Qatar changed course. The vessels were going to the UK, and when they shifted direction, Bloomberg reported that prices started to shift upward almost instantly. 

Analysis from TransitionZero, reported by The Guardian, says that the costs associated with natural gas are now so high that it’s no longer worth investing in as a “transition fuel.” It says that the cost to switch from coal to gas is around $235 per ton of CO2, compared to just $62 for renewables as well as the necessary battery storage.

Swearing off gas

Stove. Cook stove. Modern kitchen stove with blue flames burning.
MarianVejcik via Getty Images

In order to break up with gas in my own home, I’ll need to swap out my stovetop (not so hard) and my whole central heating system (pretty hard). The former I can likely achieve for a few hundred dollars, plus or minus the cost of installation. (Some units just plug in to a standard wall socket, so I may be able to do much of the work myself if I’m feeling up to the task.) Of course, getting a professional to unpick the gas pipeline that connects to my stovetop is going to be harder. 

Unfortunately, replacing a 35kW condensing gas boiler (I have the Worcester Bosch Greenstar 35CDi) is going to be a lot harder. The obvious choice is an Air Source Heat Pump (ASHP), or even a geothermal Ground Source Heat Pump (GSHP), both of which are more environmentally-friendly. After all, both are more energy-efficient than a gas boiler, and both run on electricity which is theoretically cleaner.

More generally, the UK’s Energy Saving Trust, a Government-backed body with a mission to advocate for energy efficiency, says that the average Briton should expect to pay between £7,000 and £13,000 to install an ASHP. Much of that figure is dependent on how much of your home’s existing hardware you’ll need to replace. A GSHP is even more expensive, with the price starting at £14,000 and rising to closer to £20,000 depending on both your home’s existing plumbing and the need to dig a bore hole outside. 

In my case, heat pump specialists told me that, give or take whatever nasties were found during installation, I could expect to pay up to £27,000 ($33,493). This included a new ASHP, radiators, hot water and buffer cylinders, pumps, piping, controllers, parts and labor. Mercifully, the UK is launching a scheme to offer a £5,000 ($6,200) discount on any new heat pump installations. But that still means that I’m paying north of £20,000 (and ripping out a lot of existing materials with plenty of life left in them) to make the switch. 

In the US, there’s plenty of difference on a state level, but at the federal level, you can get a tax credit on the purchase of a qualifying GSHP. A system installed before January 1st, 2023, will earn a 26 percent credit, while a unit running before January 1st, 2024, will be eligible for a 22 percent credit. Purchasers of a qualifying ASHP, meanwhile, were entitled to a $300 tax credit until the end of 2021. 

The contractors also provided me with a calculation of my potential energy savings over the following seven years. It turns out that I’d actually be spending £76 more on fuel per month, and £532 over the whole period. On one hand, if I had the cash to spare, it’s a small price to pay to dramatically reduce my personal carbon emissions. On the other, I was hoping that the initial investment would help me reduce costs overall, but that's not the case while the cost of gas is (ostensibly) cheaper than electricity. (This will, of course, change as energy prices surge in 2023, however, but I can only look at the data as it presently stands.)

An aside: To be honest with you all, I was fully aware that the economic case for installing a heat pump was always going to be a shaky one. When speaking to industry figures last year, they said that the conversation around “payback” isn’t shared when installing standard gas boilers. It doesn’t help that, at present, levies on energy mean that natural gas is subsidized more than energy, disincentivizing people making the switch. The rise of electric cars, too, has meant that demand for power is going to increase sharply as more people switch, forcing greater investment in generation. What’s required just as urgent is a series of measures to promote energy efficiency to reduce overall demand for both gas and electricity. 

Energy efficiency

LONDON, ENGLAND - JULY 14: Grand Design's Kevin McCloud holds a saw beside a mock-up insulated loft during a Green Home Refurbishment Programme photocall, outside Parliament on July 14, 2009 in London, England. The TV presenter is making a case to the government to launch a nationwide green refurbishment programme by encouraging people to insulate their homes properly. (Photo by Dan Kitwood/Getty Images)
Dan Kitwood via Getty Images

The UK has had an on-again, off-again relationship with climate change mitigation measures, which has helped sow the seeds of this latest crisis. The country, with low winter temperatures, relies almost exclusively on natural gas to heat its homes, its largest energy-consuming sector. As I reported last year, around 85 percent of UK homes are heated by burning natural gas in domestic boilers. 

Work to reduce the UK’s extraordinary demand for natural gas was sabotaged by government in 2013. In 2009, under the previous Labour government, a series of levies on energy companies were introduced under the Community Energy Saving Programme. These levies were added to domestic energy bills, with the proceeds funding works to install wall or roof insulation, as well as energy-efficient heating systems and heating controllers for people on low incomes. The idea was to reduce demand for gas by making homes, and the systems that heated them, far more efficient since most of the UK’s housing stock was insufficiently insulated when built. 

But in 2013, then-Conservative-Prime Minister David Cameron was reportedly quoted as saying that he wanted to reduce the cost of domestic energy bills by getting “rid of all the green crap.” At the time, The Guardian reported that while the wording was not corroborated by government officials, the sentiment was. Essentially, that meant scrapping the levies, which at the time GreenBusinessWatch said was around eight percent of the total cost of domestic energy. Cameron’s administration also scrapped a plan to build zero-carbon homes, and effectively banned the construction of onshore windfarms which would have helped reduce the cost of domestic electricity generation. 

In 2021, the UK’s Committee on Climate Change examined the fallout from this decision, saying that Cameron’s decision kneecapped efforts to reduce demand for natural gas. As Carbon Brief highlighted at the start of 2022, in 2012, there were nearly 2.5 million energy efficiency improvements installed. By 2013, that figure had fallen to just 292,593. The drop off, the Committee on Climate Change believes, has caused insulation installations to fall to “only a third of the rate needed by 2021” to meet the national targets for curbing climate emissions. 

Carbon Brief’s report suggests that the financial savings missed by the elimination of these small levies – the “green crap,” – has cost UK households around £2.5 billion. In recent years, a pressure group – Insulate Britain – has undertaken protests at major traffic intersections to help highlight the need for a new retrofit program to be launched. The current government’s response to their pleas has been to call for tougher criminal penalties for protesters including a jail term of up to six months.

Chart from Carbon Brief in lieu of broken embed.
A chart, courtesy of Carbon Brief, showing the impact of the removal of the 'green crap' levies on domestic energy-efficiency installations in the UK.
Carbon Brief

Making my own power

Setting up of solar panels on the roof of a farm shed, used to produce electricity. (Photo by: Andia/Universal Images Group via Getty Images)
Andia via Getty Images

Looking back through my energy bills over the last few years, my household’s annual electricity consumption is around 4,500kWh per year. A heat pump would likely add a further 6,000kWh to my energy bill, not to mention any additional cost for switching to all-electric cooking. It would be sensible to see if I could generate some, or all, of my own energy at home using solar panels to help reduce the potential bill costs. 

The Energy Saving Trust says that the average homeowner can expect to pay £6,500 for a 4.2kWp system on the roof of their home. Environmental factors such as the country you live in and orientation of your property mean you can’t be certain how much power you’ll get out of a specific solar panel, but we can make educated guesses. For instance, the UK’s Renewable Energy Hub says you can expect to get around 850kW per year out of a 1kW system. For a theoretical 5kWp system in my location, the Energy Saving Trust thinks I’ll be able to generate around 4,581kWh per year. 

Sadly, I live in an area where, even though my roof is brand new and strong enough to take panels, they aren’t allowed. This is because it is an area of “architectural or historic interest where the character and appearance [of the area] needs to be protected or improved.” Consequently, I needed to explore work to ground-mount solar panels in my back garden, which gets plenty of sunlight. 

While I expected grounded panel installations to be much cheaper, they apparently aren’t. Two contractors I spoke to said that while their average roof-based installation is between £5,000 and £7,000, a 6kWp system on the ground would cost closer to £20,000. It would be, in fact, cheaper to build a sturdy shed in the bit of back yard I had my eye on and install a solar system on top of there, compared to just getting the mounting set up on the ground. That’s likely to spool out the cost even further, and that’s before we get to the point of talking about battery storage. 

The bill

many identical money notes in a mess
undefined undefined via Getty Images

For this rather nifty thought experiment, the cost for me to be able to walk away from natural gas entirely would be north of £30,000 ($37,000). Given that the average UK salary is roughly £38,000, it’s a sum that is beyond the reach of most people without taking out a hefty loan. This is, fundamentally, why the need for government action is so urgent, since it is certainly beyond the ability of most people to achieve this change on their own. 

In fact, it’s going to require significant movement from central government not just in the UK but elsewhere to really shake our love-hate relationship with natural gas. Unfortunately, given that it’s cheap, cleaner than coal and the energy lobby has plenty of muscle behind it, that’s not likely to happen soon. And so we’re stuck in a trap – it’s too expensive to do it ourselves (although that’ll certainly be an interesting experiment to undertake) and there’s no help coming, despite the energy crisis that’s unfurling around us.

Climate scientists reconsider the meaning and implications of drought in light of a changing world [Published articles]

Maps of the American West have featured ever darker shades of red over the past two decades. The colors illustrate the unprecedented drought blighting the region. In some areas, conditions have blown past severe and extreme drought into exceptional drought. But rather than add more superlatives to our descriptions, one group of scientists believes it's time to reconsider the very definition of drought.

QDB - 7634 [Published articles]

<SimonSapin> nox: the history of packaging in python is super complicated
<nox> SimonSapin: All I need to know is, is setuptools old stuff or new stuff?
<SimonSapin> nox: its been both
<SimonSapin> in that order

RSS feeds are available for many U.S. Government collections [Published articles]

Minecraft as a Kubernetes admin tool [Published articles]

The History and Politics of Wuxia [Published articles]

I first fell in love with wuxia when I was around eight or so. I remember running around swinging the bright yellow handle of my toy broom as a sword, calling a sprawling tiger stuffed toy my master and pretending the shower was a waterfall I could learn the secrets of the universe under. I ran on tiptoe because that was somehow more like flying—or “hing gung” 輕功, the art of lightness, as I would eventually become fond of translating it .

But even before then I was deeply familiar with the genre; its many conventions have become baked into the everyday language of the Hong Kong I grew up in. My relatives all played Mahjong and much like with sports, discussions around these games borrowed heavily from the language of sparring martial artists. I’d ask at the end of every Sunday, what are the results of the battles. When asking for a family recipe, someone would joke that they’d have to become the apprentice of this or that auntie. Later, there was the world of study guides and crib sheets, all calling themselves secret martial arts manuals. The conventions around martial artists going into seclusion to perfect their craft and going mad in the pursuit of it take on new meaning as slang around cramming for exams.

Which is all to say, I really love wuxia.

“Wuxia”, literally meaning “martial hero”, is a genre about martially powerful heroes existing in a world parallel to and in the shadows of the Chinese imperial history.

The archetypal wuxia hero is someone carving out his own path in the world of rivers and lakes, cleaving only to their own personal code of honour. These heroes are inevitably embroiled in personal vengeance and familial intrigue, even as they yearn for freedom and seek to better their own skills within the martial arts. What we remember of these stories are the tournaments, the bamboo grove duels and the forbidden love.

Parallels are often drawn to knights errant of medieval romances, with many older translations favouring a chivalric vocabulary. There are also obvious comparisons to be made with the American western, especially with the desperados stumbling into adventures in isolated towns in search for that ever-elusive freedom.

It is easy to think of wuxia in these universal terms with broad themes of freedom, loyalty and justice, but largely divorced from contemporary politics. These are stories, after all, that are about outlaws and outcasts, existing outside of the conventional hierarchies of power. And they certainly do have plenty to say about these big universal themes of freedom, loyalty and justice.

But this is also a genre that has been banned by multiple governments within living memory. Its development continues to happen in the shadows of fickle Chinese censorship and at the heart of it remains a certain defiant cultural and national pride intermingled with nostalgia and diasporic yearning. The vast majority of the most iconic wuxia texts are not written by Chinese authors living comfortably in China, but by a dreaming diaspora amid or in the aftermath of vast political turmoil.

Which is all to say that the world of wuxia is fundamentally bound up with those hierarchies of power it seeks to reject. Much like there is more to superheroes than dorky names, love triangles, and broad universal ideals of justice, wuxia is grounded in the specific time and place of its creation.

Biography of Old Dragon-beard (虯髯客傳) by Du Guangting (杜光庭, 850-933) is commonly cited as the first wuxia novel. It chronicles the adventures of the titular Old Dragon-beard, who along with the lovers, Hongfu 紅拂 and Li Jing 李靖, make up the Three Heroes of the Wind and Dust. But the story isn’t just supernatural adventures; they also help Li Shimin 李世民 found the Tang Dynasty (618–906). The martial prowess and the seemingly eccentric titles of the characters aside, the act of dynastic creation is unavoidably political. 虯髯客傳 pivots around Hongfu’s ability to discern the true worth a man, which leads her to abandon her prior loyalties and cleave her love to Li Jing and his vision for a better empire. Not to mention Du wrote this and many of his other works whilst in exile with the Tang imperial court in the south, after rebels sacked the capital and burnt his books. Knowing this, it is difficult not to see Du as mythologising the past into a parable of personal resonance, that perhaps he too was making decisions about loyalties and legacies, which court or emperor he should stay with, asking himself if the Tang would indeed rise again (as he himself, as a taoist has prophecised).

Other commonly cited antecedents to the modern wuxia genre are the 14th Century classics like Romance of the Three Kingdoms (三國演義) and Outlaws of the Marsh (水滸傳), the former of which is all about the founding of dynasties and gives to Chinese the now ubiquitously cited The empire, long divided, must unite; long united, must divide. Thus it has ever been (话说天下大势.分久必合,合久必分).

 

Revolutionaries, Rebels and Race in the Qing Dynasty

No era of imperial China was in possession of a “free press”, but the literary inquisitions under the Qing Dynasty (1644–1911) were particularly bloody and thorough. The Manchu elite suppressed any openly revolutionary sentiment in fiction, however metaphorical, and what is written instead is a literature that sublimates much of that discontent into historical fiction nostalgic for the eras of Han dominance. Wandering heroes of the past were refashioned into a pariah elite, both marginalised from mainstream society but also superior to it with their taoist-cultivated powers.

Whilst earlier quasi-historical epics and supernatural tales are replete with gods and ghosts, late Qing wuxia begins to shed these entities and instead grounds itself in a world where taoist self-cultivation grants immense personal powers but not divinity itself. In each of the successive reprintings of Three Heroes and Five Gallants (三俠五義), editors pruned the text of anachronisms and supernatural flourishes.

The parallel world of secret societies, foreign cults, bickering merchants and righteous martial clans came to be known as jianghu, literally “rivers and lakes”. As a metaphor, it was first coined by taoist philosopher, Zhuangzi 莊子, to describe a utopian space outside of cutthroat court politics, career ambitions and even human attachments. This inspires subsequent generations of literati in their pursuits of aesthetic hermitism, but the jianghu we know today comes also from the waterways that form the key trade routes during the Ming Dynasty (1368–1644). To the growing mercantile classes, jianghu referred to the actual rivers and canals traversed by barges heavy with goods and tribute, a byname for the prosperous Yangtze delta.

These potent lineages of thought intermingle into what jianghu is within martial arts fiction today, that quasi historical dream time of adventure. But there is also another edge to it. In Stateless Subjects: Chinese Martial Arts History and Postcolonial History, Petrus Liu translates jianghu as “stateless”, which further emphasizes that the hero’s rejection of and by the machineries of government. Jianghu is thus a world that rejects the dictates of the state in favor of divine virtue and reason, but also of a sense of self created through clan and community.

The name of the genre, wuxia (“武俠“) comes from Japanese, where a genre of martially-focused bushido-inspired fiction called bukyō (“武侠”) was flourishing. It was brought into Chinese by Liang Qichao 梁启超, a pamphleteer writing in political exile in Japan, seeking to reawaken what he saw as Han China’s slumbering and forgotten martial spirit. In his political work, he holds up the industrialisation and militarisation of Meiji Japan (and its subsequent victory against Russia) as inspiration and seeks a similar restoration of racial and cultural pride for the Han people to be the “master of the Continent” above the hundred of different races who have settled in Asia.

Wuxia is fundamentally rooted in these fantasies of racial and cultural pride. Liang Qichao’s visions of Han exceptionalism were a response to subjugation under Manchu rule and Western colonialism, a martial rebuttal to the racist rhetoric of China being the “Sick Man of Asia”. But it is still undeniably ethno-nationalism built around the descendants of the Yellow Emperor conquering again the continent that is their birthright. Just as modern western fantasy has as its bones the nostalgia for a pastoral, premodern Europe, wuxia can be seen as a dramatisation of Sinocentric hegemony, where taoist cultivation grants power and stalwart heroes fight against an ever-barbaric, ever-invading Other.

 

Dreams of the Diaspora

Jin Yong 金庸 remains synonymous with the genre of wuxia in Chinese and his foundational mark on it cannot be overstated. His Condor Trilogy (射鵰三部曲) was serialised between 1957-63 and concerns three generations of heroes during the turbulent 12th-13th centuries. The first concerns a pair of sworn brothers, one loyal and righteous, the other clever and treacherous. Their friendship deteriorates as the latter falls into villainy, scheming with the Jin Empire (1115–1234) to conquer his native land. The second in the trilogy follows their respective children repeating and atoning for the mistakes of their parents whilst the Mongols conquer the south. The last charts the internal rivalries within the martial artists fighting over two peerless weapons whilst its hero leads his secret society to overthrow the Yuan Dynasty (1271–1368).

It’s around here that English articles about him start comparing him to Tolkien, and it’s not wholly unjustified, given how both created immensely popular and influential legendaria that draw heavily upon ancient literary forms. Entire genres of work have sprung up around them and even subversions of their work have become themselves iconic. Jin Yong laid down what would become the modern conventions of the genre, from the way fights are imagined with discrete moves, to the secret martial arts manuals and trap-filled tombs.

Buy it Now

Unlike Tolkien, however, Jin Yong’s work is still regularly (even aggressively) adapted. There are in existence nine tv adaptations of each instalment of the Condor Trilogy, for example, as well as a video game and a mobile game. And at time of writing, eight feature films and nine tv series based on his work are in production.

But Jin Yong’s work was not always so beloved by mainland Chinese audiences. For a long time he, along with the rest of wuxia, were banned and the epicentre of the genre was in colonial Hong Kong. It is a detail often overlooked in the grand history of wuxia, so thoroughly has the genre been folded into contemporary Chinese identity. It is hard at times to remember how much of the genre was created by these artists in exile. Or perhaps that is the point, as Hong Kong’s own unique political and cultural identity is being subsumed into that of the People’s Republic, so too is its literary legacy. Literalist readings of his work as being primarily about historical martial artists defang the political metaphors and pointed allegories.

Jin Yong’s work is deeply political. Even in the most superficial sense, his heroes intersect with the politics of their time, joining revolutionary secret societies, negotiating treaties with Russia and fighting against barbarian invaders. They are bound up in the temporal world of hierarchy and power. Legend of the Condor Hero (射鵰英雄傳)’s Guo Jing 郭靖 becomes the sworn brother to Genghis Khan’s son, Tolui, and joins the Mongol campaign against the Khwarezmid Empire. Book and Sword (書劍恩仇錄)’s Chen Jialuo 陳家洛 is secretly the Qianlong Emperor’s half brother. The Deer and the Cauldron (鹿鼎記)’s Wei Xiaobao 韋小寶 is both best friends with the Kangxi Emperor and also heavily involved in a secret society dedicated to overthrowing the aforementioned emperor. Even Return of the Condor Hero (神鵰俠侶)‘s Yang Guo 楊過 ends up fighting to defend the remains of the Song Empire against the Mongols.

But it goes deeper than that. Jin Yong was a vocal critic of the Cultural Revolution, penning polemics against Mao Zedong and the Gang of Four during the late 60s. Beyond the immediate newspaper coverage, Jin Yong edited and published many more works both documenting and dissecting the Cultural Revolution.

Jin Yong described himself as writing every day one novel instalment and one editorial against the Gang of Four. Thus did they bleed together, the villains of Laughing in the Wind (笑傲江湖) becoming recognisable caricatures as it too rejected senseless personality cults.

In this light, his novels seem almost an encyclopaedia of traditional Chinese culture, its values and virtues, a record of it to stand bulwark against the many forces that would consign it all to oblivion. It is a resounding rebuttal to principles of the May Fourth Movement, that modernisation and westernisation are equivalents. To Jin Yong the old and the traditional were valuable, and it is from this we must build our new literature .

Taken together, Jin Yong’s corpus offers an alternate history of the Han people spanning over two thousand years from the Eastern Zhou (771–256 B.C.) to the Qing Dynasty (1644–1911). He fills in the intriguing gaps left in official records with folk heroes, court gossip and conspiracy theories. His text is dense with literary allusions and quotations from old Chinese poems.

His stories are almost all set during times of turmoil when what can be termed “China”, or at least, the Han people are threatened by barbarian invasion and internal corruption; pivotal moments in history that makes heroes and patriots out of ordinary men and women. All this Jin Yong immortalises with a deep yearning for a place and past that never quite was; nostalgia in the oldest sense of the word, with all the pain and pining and illusion that it implies.

It is arguably this very yearning, this conjuring of a real and relevant past from dry history books that makes Jin Yong’s work so endlessly appealing to the Chinese diaspora, as well as the mainland Chinese emerging from the Cultural Revolution. This alternate history dramatises the complexities of Han identity, all the times it has been threatened, disrupted and diluted in history, but at the same time it gave hope and heroics. These were stories as simple or as complex as the reader wanted it to be.

 

Chinese Imperialism and Han Hegemony

It is sometimes hard to remember that Jin Yong and all the rest of wuxia was once banned in the People’s Republic of China, so thoroughly have they now embraced his work. As late as the 1990s was Jin Yong decried as one of the “Four Great Vulgarities of Our Time” (alongside the four heavenly kings of cantopop, Jackie Chan and sappy Qiong Yao romances).

In recent decades, the CCP has rather dramatically changed its relationship with the past. The censorship machine is still very active, but it does not have in its crosshairs the decadent and feudal genre of wuxia (though there have been exceptions, especially during the run up to the Republic’s 70th anniversary when all frivolous dramas were put on pause; it is important to remember that the censors are not always singular or consistent in their opinions). But more importantly, the Party no longer draws power from a radical rejection of the past, instead it is embraces utterly, celebrated at every turn. Traditionalism now forms a core pillar of their legitimacy, with all five thousand years of that history validating their rule. The State now actively promotes all those superstitions and feudal philosophies it once held in contempt.

Along with the shifting use of history to inspire nationalism has Jin Yong been rehabilitated and canonised. It’s arguably that revolutionary traditionalism —that he was preserving history in a time of its destruction—that makes him so easy to rehabilitate. Jin Yong’s work appeals both to the conservative mind with its love of tradition and patriotic themes, but also to rebels in its love of outlaw heroes.

It isn’t that these stories have nothing to say on themes of a more abstract or universal sense of freedom or justice, but that they are also very much about the specifics of Han identity and nationalism. Jin Yong’s heroes often find themselves called to patriotism, even as they navigate their complex or divided loyalties, they must defend “China” in whatever form it exists in at the time against barbaric, alien invaders. Even as they function as straightforward stories of nationalistic defence, they are also dramatising disruptions of a simplistic or pure Chinese identity, foregrounding characters from marginalised (if also often exoticised) ethnicities and religions.

Jin Yong’s hero Guo Jing is Han by birth and Mongol by adoption. He ultimately renounces his loyalty to Genghis Khan and returns to his Han homeland to defend it from Mongol conquest. Whilst one can read Jin Yong’s sympathy and admiration for the Mongols as an attempt to construct an inclusive nationalism for modern China, Guo Jing’s participation as a Han hero in the conquest of Central Asia also functions as a justification of modern Han China’s political claim on that imperial and colonial legacy.

Book and Sword has this even more starkly as it feeds the popular Han fantasy that the Kangxi Emperor is not ethnically Manchu but instead, a Han changeling. He is forced by the hero of the novel Chen Jialuo to swear an oath to acknowledge his Han identity and overthrow the Manchus, but of course, he then betrays them and subjugates not only the Han but also the “Land of Wei” (now known as Xin Jiang, where the genocide is happening). Still there is something to be said about how this secret parentage plot attributes the martial victories of the Qing to Han superiority and justifies the Han inheritance of former Qing colonies.

The Uyghur tribes are portrayed with sympathy in Book and Sword. They are noble and defiant and devout. Instead of savages who need to be brought to heel, they are fellow resistance fighters. It alludes to an inclusive national identity, one in which Han and Uyghur are united by their shared suffering under Manchu rule. It can also be argued that their prominence disrupts the ideal of a pure Han-centric Chineseness. But what good is inclusion and unity to those who do not want to be part of that nation? Uyghurs, being a people suffering occupation, actively reject the label of “Chinese Muslims”.

Furthermore, the character of Kasili in Book and Sword, based on the legend of the Fragrant Concubine, is drenched in orientalist stereotype. Chen first stumbles upon her bathing naked in a river, her erotic and romantic availability uncomfortably paralleling that of her homeland. When the land of Wei falls to the emperor’s sword and Kasili is taken as a concubine, she remains loyal to the Han hero she fell in love with, ultimately killing herself to warn Chen of the emperor’s duplicity. Conquest and imperial legacy is thus dramatised as a love triangle between a Uyghur princess, a Han rebel and a Manchu emperor.

Chen, it should be noted, falls in love and marries a different Uyghur princess for his happy ending.

Amid other far more brutal policies meant to forcibly assimilate and eradicate Uyghur identity, the PRC government encouraged Han men to take Uyghur women as wives. Deeply unpleasant adverts still available online extolled the beauty and availability of Uyghur women, as something and somewhere to be conquered. It is impossible not to be reminded of this when reading about the beautiful and besotted Kasili.

There is no small amount of political allegory to be read between the lines of Jin Yong, something he became increasingly frank about towards the end of his life. Condor Trilogy with its successive waves of northern invaders can be seen as echoing at the Communist takeover of China. The success of Wei Xiaobao’s affable cunning can be a satire on the hollowness materialistic 70s modernity. But Jin Yong himself proved to be far less radical than his books as he sided with the conservative anti-democracy factions within Hong Kong during the Handover.

In an 1994 interview, Jin Yong argues against the idea that China was ever under “foreign rule”, instead proposing that the many ethnic groups within China are simply taking turns on who happens to be in ascendance. All wars are thus civil wars and he neatly aligns his novels with the current Chinese policies that oppress in the name of unity, harmony and assimilation, of “inclusive” nationalism.

The legacy of Jin Yong is a complex one. His work, like all art, contains multitudes and can sustain any number of seemingly contradictory interpretations. It is what is beautiful about art. But I cannot but feel that his rapid canonisation over the last decades in mainland China is a stark demonstration of how easily those yearning dreams of the diaspora can become nationalistic fodder.

 

In Closing

I did not come to bury wuxia, but to praise it. I wanted to show you a little bit of its complexities and history, as well as the ideals and ideologies that simmer under its surface.

For me, I just think it is too easy to see wuxia as a form of salvation. Something to sustain and inspire me in a media landscape hostile to people who look like me. To give me the piece of me that I have felt missing, to heal a deep cultural wound. After all, Hollywood or broader Anglophone media might be reluctant to make stories with Asian protagonists, but I can turn to literally all of wuxia. American TV series won’t make me a fifty episode epic about two pretty men eyefucking each other that also has a happy ending, but I will always have The Untamed.

It’s this insidious feeling of hope. That this genre is somehow wholly “unproblematic” because I am reconnecting with my cultural roots, that it can nourish me. That it can be safe that way. It is, after all, untouched by all the problematic elements in Anglophone mainstream that I have analysed to death and back. That it is some sort of oasis, untouched by colonialism and western imperialism. That it therefore won’t or can’t have that taint of white supremacy; it’s not even made by white people.

Perhaps it is just naive of me to have ever thought these things, however subconsciously. Articulating it now, it’s ridiculous. Han supremacy is a poisonous ideology that is destroying culture, hollowing out communities and actively killing people. In the face of its all-consuming genocide-perpetuating ubiquity, the least I can do is recognise its presence in a silly little genre I love. It just doesn’t seem too much to ask.

Jeannette Ng is originally from Hong Kong but now lives in Durham, UK. Her MA in Medieval and Renaissance Studies fed into an interest in medieval and missionary theology, which in turn spawned her love for writing gothic fantasy with a theological twist. She runs live roleplay games and is active within the costuming community, running a popular blog. Jeannette has been a finalist for the John W. Campbell Award for Best New Writer and the Sydney J Bounds Award (Best Newcomer) in the British Fantasy Awards 2018.

Report Finds Phone Network Encryption Was Deliberately Weakened [Published articles]

A weakness in the algorithm used to encrypt cellphone data in the 1990s and 2000s allowed hackers to spy on some internet traffic, according to a new research paper. Motherboard: The paper has sent shockwaves through the encryption community because of what it implies: The researchers believe that the mathematical probability of the weakness being introduced on accident is extremely low. Thus, they speculate that a weakness was intentionally put into the algorithm. After the paper was published, the group that designed the algorithm confirmed this was the case. Researchers from several universities in Europe found that the encryption algorithm GEA-1, which was used in cellphones when the industry adopted GPRS standards in 2G networks, was intentionally designed to include a weakness that at least one cryptography expert sees as a backdoor. The researchers said they obtained two encryption algorithms, GEA-1 and GEA-2, which are proprietary and thus not public, "from a source." They then analyzed them and realized they were vulnerable to attacks that allowed for decryption of all traffic. When trying to reverse-engineer the algorithm, the researchers wrote that (to simplify), they tried to design a similar encryption algorithm using a random number generator often used in cryptography and never came close to creating an encryption scheme as weak as the one actually used: "In a million tries we never even got close to such a weak instance," they wrote. "This implies that the weakness in GEA-1 is unlikely to occur by chance, indicating that the security level of 40 bits is due to export regulations." Researchers dubbed the attack "divide-and-conquer," and said it was "rather straightforward." In short, the attack allows someone who can intercept cellphone data traffic to recover the key used to encrypt the data and then decrypt all traffic. The weakness in GEA-1, the oldest algorithm developed in 1998, is that it provides only 40-bit security. That's what allows an attacker to get the key and decrypt all traffic, according to the researchers.

Read more of this story at Slashdot.

An Indian art form called Rangoli [Published articles]

Fact or fantasy? Tales from the linguistic fringe [Published articles]

November 2020 SaltStack CVEs: CVE-2020-16846, CVE-2020-17490, CVE-2020-25592 [Published articles]

SaltStack has released a security update to Salt to address three critical vulnerabilities. We strongly recommend that you prioritize this update.

This is a security release. The following CVE’s were fixed as part of this release:

  • CVE-2020-16846
  • CVE-2020-17490
  • CVE-2020-25592

Read more

Sandro Tosi: Multiple git configurations depending on the repository path [Published articles]

For my work on Debian, i want to use my debian.org email address, while for my personal projects i want to use my gmail.com address.

One way to change the user.email git config value is to git config --local in every repo, but that's tedious, error-prone and doesn't scale very well with many repositories (and the chances to forget to set the right one on a new repo are ~100%).

The solution is to use the git-config ability to include extra configuration files, based on the repo path, by using includeIf:

Content of ~/.gitconfig:

[user]
name = Sandro Tosi
email = <personal.address>@gmail.com

[includeIf "gitdir:~/deb/"]
path = ~/.gitconfig-deb

Every time the git path is in ~/deb/ (which is where i have all Debian repos) the file ~/.gitconfig-deb will be included; its content:

[user]
email = morph@debian.org
That results in my personal address being used on all repos not part of Debian, where i use my Debian email address. This approach can be extended to every other git configuration values.

All DuckDuckGo bang operators on one page [Published articles]

Some Countries Reopened Schools. What Did They Learn About Kids and Covid? [Published articles]

Studies from around the world suggest that success depends on class size, distancing, the age of the students, and how prevalent the virus is locally.

An Acceptance, in rough times [Published articles]

Last night, starting at at 1:00 in the morning, my time, was the Nebula Awards ceremony, held by the SFWA, the Science Fiction and Fantasy Writers of America. The first award they gave out was the Ray Bradbury Award for Outstanding Dramatic Presentation, and it meant the world that it went to episode 3 of Good Omens, "Hard Times".

Exactly one year ago, Good Omens was released to the world, on Amazon's Prime Video service. Thirty years ago this month, Good Omens was published as a novel. It seems amazing that it still has so much life, and still feels so relevant to people's own lives. Especially now.

Here's the complete list of all the nominees and of the awards given out at the Nebulas last night. Congratulations to everyone nominated!

The entire proceedings existed in virtual space, via the magic of Zoom and other technological things.
This is what it looked like on my screen, just before we went live...


Here is the speech I gave. I wore a hat, because, even though Terry Pratchett loved pointing out that he was a hat person and I wasn't, not really, I thought it would have amused him.

I didn't intend to write the television adaptation of Good Omens. I did it because as he knew his own immeasurable light was dimming, Terry Pratchett wrote to me, telling me I had to do it. That no-one else had the passion for the “old girl” that the two of us had. And I was the one of us who had to make it happen, so he could see it before the lights went out.

I'm used to dealing with the problems of fictional people.  Now I found myself dealing with much harder problems, of real people and immutable budgets.  But I was even more determined to make something Terry would have been proud of. And I was part of an amazing team – Douglas Mackinnon, our director, Rob Wilkins, Chris Sussman and Simon Winstone and the folk from BBC Studios, the Amazon Studios team, and above us all the cast and the crew, who united and went over and above what anyone asked of them to tell, together, a kind of love story about protecting the world, about an angel who isn't as angelic as he ought to be, and a demon who likes people. And for them, I want to thank Michael Sheen and David Tennant.

Terry and I had written a book about averting the end of the world, about the power of not going to war, about an armageddon that didn't have to happen.

When I was a boy, I was told that there was a curse, “May you live in interesting times”. And that made me sad, because I wanted to live in interesting times. I thought I did.

And now, we are all of us living in Interesting Times. The Horsepeople are riding out, as they have ridden so many times before, and the world still needs saving – from plague, from racism, from foolishness and selfishness and pain. It says in Good Omens that we have to save ourselves, because nobody else is going to sort it out for us. And we do. 

It feels almost indecent to be accepting an award while so many people are hurting, but thank you, from me and from Douglas, who took the words and made them so brilliantly come to life. This is for Terry Pratchett.

You can watch the whole ceremony at: 

https://www.facebook.com/SFWA.org/videos/996082517476423/

or at this YouTube link:
  


(The Good Omens bit starts around 22:30)



My list of underrated movie scores and themes... Part 1 [Published articles]

Been putting this together for a while... more to come.
In no particular order, though grouped by composer.
To be clear, I'm in no way saying these are unknown themes or not loved. In my limited experience, they just don't get the same acclaim as some more well-known scores, and I feel they deserve recognition! These are just pieces of music uncannily suited to their films, and work perfectly in the movie while also standing alone as wonderful pieces of music.

And while I haven't completely steered away from the John Williams' and Jerry Goldsmiths of the world, I have tried to include slightly more off-kilter selections that are truly fantastic.

Basil Poledouris
Klendathu Drop - Starship Troopers

https://www.youtube.com/watch?v=CIGHCoVzqtk&list=PLVWEHrcKGof5WwMraVM42fIVA_vqJd1XH&index=2&t=0s

Robocop Theme - Robocop

https://www.youtube.com/watch?v=Kb1_38445vA&list=PLVWEHrcKGof5WwMraVM42fIVA_vqJd1XH&index=3&t=0s

Riddle of Steel & Riders of Doom - Conan the Barbarian

https://www.youtube.com/watch?v=EAFtiUoq6TE&list=PLVWEHrcKGof5WwMraVM42fIVA_vqJd1XH&index=4&t=0s

Ennio Morricone

Love Theme - Cinema Paradiso

https://www.youtube.com/watch?v=osOE3trekhg&list=PLVWEHrcKGof5WwMraVM42fIVA_vqJd1XH&index=9&t=0s

Complete Score - The Thing

https://www.youtube.com/watch?v=zgiSXRoG2tQ

Ecstasy of Gold - The Good, The Bad, and The Ugly

https://www.youtube.com/watch?v=enuOArEfqGo

Bill Conti

Going The Distance & The Final Bell - Rocky

https://www.youtube.com/watch?v=knOrIBEqxYc&list=PLVWEHrcKGof5WwMraVM42fIVA_vqJd1XH&index=14&t=0s

Main Theme - The Right Stuff

https://www.youtube.com/watch?v=tCblQ_fnPpc&list=PLVWEHrcKGof5WwMraVM42fIVA_vqJd1XH&index=15&t=0s

Jerry Goldsmith

Main Theme - Capricorn One

https://www.youtube.com/watch?v=aXiNHJ5TANs&list=PLVWEHrcKGof5WwMraVM42fIVA_vqJd1XH&index=17&t=0s

Main Theme - Gremlins II (and Gremlins... just a great performance of it)

https://www.youtube.com/watch?v=3yxxSU0nw64

Main Title - Planet of the Apes

https://www.youtube.com/watch?v=a1rq6NoddX0

The Enterprise - Star Trek: The Motion Picture

https://www.youtube.com/watch?v=w1Cvyrvxlbk

Erich Wolfgang Korngold

Main Title - Kings Row (also... the inspiration for Star Wars...)

https://www.youtube.com/watch?v=tysCiL1-24w

Main Title - Reunion - The Sea Hawk

https://www.youtube.com/watch?v=hSkA-Ntst5w

John Williams

Main Theme - Seven Years in Tibet (one of his best)

https://www.youtube.com/watch?v=H3--WW2cNYY

Main Theme - Born on the Fourth of July

https://www.youtube.com/watch?v=ELKzmZK5Dec

With Malice Towards None - Lincoln
https://www.youtube.com/watch?v=BqDIgf5THbk&list=PLVWEHrcKGof5WwMraVM42fIVA_vqJd1XH&index=21&t=0s

Alan Silvestri

Main Theme - Predator

https://www.youtube.com/watch?v=9c3biXCLUZY

Main Theme - Contact (Maybe my fav on the list... I'm a sucker for sentimentality... Sue me)

https://www.youtube.com/watch?v=OrqWF2GmXE4

Captain America March - Captain America: The First Avenger

https://www.youtube.com/watch?v=qrXwAeJ87Bk

MISC
Junkie Xl - Mad Max: Fury Road

https://www.youtube.com/watch?v=U3Mg8GJpVfo

Daft Punk - Tron Legacy

https://www.youtube.com/watch?v=qolzPKVQa10

James Horner - Commando

https://www.youtube.com/watch?v=REb3vFSkQEg

submitted by /u/theblackshell to r/moviemusic
[link] [comments]

'How to build a Nintendo Switch' for coronavirus #StayAtHome gaming [Published articles]

Wow, man. Some of us take on more extreme projects during the Great Coronavirus Quarantine than others.

This ambitious fellow shows you how to build a Nintendo Switch, with a beautiful and wholesome purpose: “to Starve Online Price Gougers” who are jacking up the prices because demand is high for Nintendo Switch, and availability is nil.

Here's their introduction to the HOWTO gallery, which is amazing and stupendous.

After playing New Horizons and hyping it up to my friends, they decided they wanted a Switch. They called around to different retailers every day for a week with no luck finding anyone who had one in stock. No one knew when the next shipment would be. This led to an online search like Craigslist, OfferUp, and Ebay.

Unfortunately everyone knows the rest. Upwards of $450 to $600 in the Seattle area for a used Switch. Some with and without all the accessories. This enraged me to the point of telling them I could build one cheaper out of spare parts. So they hired me to do just that. If anyone is interested in doing the same here is my step by step buying guide along with assembly instructions and a pricing guide.

1. Game Cartridge Card Slot Socket Board w/Headphones Port - $15
2. NS Console Micro SD TF Memory Card Slot Port Socket Reader - $5
3. Nintendo Switch HAC-001 CPU Cooling Heatsink - $7
4. Game Cartridge Card Plastic Cover - $1
5. Console Speaker Replacement Parts For Nintendo Switch Built in speaker - $8
6. Wifi Antenna Connecting Cable (Short) $2
7. Wifi Antenna Connecting Cable (Long) $2
8. Internal Cooling Fan - $3
9. Power & Volume Button control flex cable (w/ buttons and rubber conductor) - $4
10. Side Slider Sliding Rail Flex Cable (Left) - $3
11. Side Slider Sliding Rail Flex Cable (Right) - $3
12. Replacement Top Housing Shell Case Face plate -$6
13. Nintendo Switch Console Replacement Battery (New) - $15
14. Replacement Bottom Housing Shell Transparent Case Face plate -$5
15. Touch Screen Digitizer Adhesive - $0.50
16. Touch Screen Digitizer - $9
17. LCD Display Screen Replacement - $12
18. Shield Plate - $2
19. Iron Middle Frame - $6
20. (Not Pictured Here) - 100% WORKING OEM NINTENDO SWITCH REPLACEMENT LOGIC BOARD MOTHERBOARD - $95
21. (Not Pictured Here) - Full Screw Replacement Set - $2
22. (Not Pictured Here) - (Removal of Copper Sicker on CPU)

Grand Total For Used Parts Build: = $199
Ebay Average Price Jan 2020: = (between $175 and $225)
Ebay Average Price April 2020: = (between $300 and $400)

I am sure I made made mistakes in this post so feel free to correct me if I am wrong about anything.

Good Luck!
And screw you if you are one of the bad guys making a buck off of a crisis.

Here you go...

How to Build A Nintendo Switch to Starve Online Price Gougers
by @sarbaaz37

How to Build A Nintendo Switch to Starve Online Price Gougers

Stuck at home? Entertain or educate yourself for free [Published articles]

It used to be that being a couch potato was almost universally deemed a negative—but it’s funny how it only takes a contagious epidemic to turn the normal state of things on its head. Fortunately, nobody with a computer need be without ways to occupy their  time.

Publishers, studios, and other media agencies are providing free offerings to give people plenty to do to ride out the corona lockdowns—as well as tools to assist self-education or learning at home. Here are a few of them I’ve noticed.

Educational/children’s book publisher Scholastic is offering a free 20-day learn-at-home program for grades K-9 via its web site—very handy for those in areas whose schools have closed down.

Would your children like to learn more about whales? Seattle-based research institute Oceans Initiative has launched a free Virtual Marine Biology Camp to teach school-closed children more about aquatic life. They’re holding live sessions every Monday and Thursday at 11 a.m. Pacific (2 p.m. Eastern) to help give those out-of-school children something educational to do.

Audiobook publisher and Amazon subsidiary Audible.com is making hundreds of audiobook titles available for free for the duration of school closures, via stories.audible.com.

NPR, the Sarasota Herald-Tribune, and CNET, among others, have articles collecting a lot of other free entertainment and education sources that weren’t free before the Corona quarantines. (Indeed, all you need do is google “coronavirus free entertainment” to find all the others who had the same idea.) But there are also still plenty of things that were already free and still are.

Baen’s Free Library is, of course, still just as free as it ever was. If you’re a member of a compatible public library, Hoopla Digital will let you borrow a limited number of ebooks, audiobooks, albums, movies, or TV episodes per month for free. And you still have access to Project Gutenberg, Librivox for audiobooks, Archive.org for all sorts of content, and all the other public-domain sites out there.

Online academic database JSTOR has over 6,000 ebooks and 150 journals that are available to the general public, and could also help to fill the education gap with schools closed down.

If you’re looking for something interesting to watch, Open Culture has links to over 200 free documentary films online, on subjects as diverse as Hayao Miyazaki and M.C. Escher. The site also includes links to free ebooks, audiobooks, online courses, and textbooks.

If you’re into anime, most of Crunchyroll‘s anime titles are available to watch for free (save for the very newest episode). Resolution may be limited, and you may have to put up with advertisements—but free is free, right? Pluto TV has over 250 channels of free video content, too, with mobile apps for iOS and Android available. And YouTube has its usual countless hundreds of thousands of hours of enjoyable ways to entertain or improve yourself, including its “Learning” category.

If you’re more into computer games, you could check out the Homecoming City of Heroes servers. Coming up on a full year since the game originally returned, it has thousands of players once again enjoying life in the early-2000s superhero MMO. (I play primarily on the Torchbearer shard, myself, and am always happy to help out new or returning players.)

There are many more free education or entertainment resources than I could even list, and there will doubtless be more the longer this lockdown goes on. How about adding your favorites in the comments?

Photo by Eric Antunes on Pexels.com


If you found this post worth reading and want to kick in a buck or two to the author, click here.

Daniel Lange: Fixing the Nextcloud menu to show more than eight application icons [Published articles]

I have been late to adopt an on-premise cloud solution as the security of Owncloud a few years ago wasn't so stellar (cf. my comment from 2013 in Encryption files ... for synchronization across the Internet). But the follow-up product Nextcloud has matured quite nicely and we use it for collaboration both in the company and in FLOSS related work at multiple nonprofit organizations.

There is a very annoying "feature" in Nextcloud though that the designers think menu items for apps at the top need to be limited to eight or less to prevent information overload in the header. The whole item discussion is worth reading as it it an archetypical example of design prevalence vs. user choice.

And of course designers think they are right. That's a feature of the trade.
And because they know better there is no user configurable option to extend that 8 items to may be 12 or so which would prevent the annoying overflow menu we are seeing with 10 applications in use:

Screenshot of stock Nextcloud menu

Luckily code can be changed and there are many comments floating around the Internet to change const minAppsDesktop = 8. In this case it is slightly complicated by the fact that the javascript code is distributed in compressed form (aka "minified") as core/js/dist/main.js and you probably don't want to build the whole beast locally to change one constant.

Basically

const breakpoint_mobile_width = 1024;

const resizeMenu = () => {
    const appList = $('#appmenu li')
    const rightHeaderWidth = $('.header-right').outerWidth()
    const headerWidth = $('header').outerWidth()
    const usePercentualAppMenuLimit = 0.33
    const minAppsDesktop = 8
    let availableWidth = headerWidth - $('#nextcloud').outerWidth() - (rightHeaderWidth > 210 ? rightHeaderWidth : 210)
    const isMobile = $(window).width() < breakpoint_mobile_width
    if (!isMobile) {
        availableWidth = availableWidth * usePercentualAppMenuLimit
    }
    let appCount = Math.floor((availableWidth / $(appList).width()))
    if (isMobile && appCount > minAppsDesktop) {
        appCount = minAppsDesktop
    }
    if (!isMobile && appCount < minAppsDesktop) {
        appCount = minAppsDesktop
    }

    // show at least 2 apps in the popover
    if (appList.length - 1 - appCount >= 1) {
        appCount--
    }

    $('#more-apps a').removeClass('active')
    let lastShownApp
    for (let k = 0; k < appList.length - 1; k++) {
        const name = $(appList[k]).data('id')
        if (k < appCount) {
            $(appList[k]).removeClass('hidden')
            $('#apps li[data-id=' + name + ']').addClass('in-header')
            lastShownApp = appList[k]
        } else {
            $(appList[k]).addClass('hidden')
            $('#apps li[data-id=' + name + ']').removeClass('in-header')
            // move active app to last position if it is active
            if (appCount > 0 && $(appList[k]).children('a').hasClass('active')) {
                $(lastShownApp).addClass('hidden')
                $('#apps li[data-id=' + $(lastShownApp).data('id') + ']').removeClass('in-header')
                $(appList[k]).removeClass('hidden')
                $('#apps li[data-id=' + name + ']').addClass('in-header')
            }
        }
    }

    // show/hide more apps icon
    if ($('#apps li:not(.in-header)').length === 0) {
        $('#more-apps').hide()
        $('#navigation').hide()
    } else {
        $('#more-apps').show()
    }
}

gets compressed during build time to become part of one 15,000+ character line. The relevant portion reads:

var f=function(){var e=s()("#appmenu li"),t=s()(".header-right").outerWidth(),n=s()("header").outerWidth()-s()("#nextcloud").outerWidth()-(t>210?t:210),i=s()(window).width()<1024;i||(n*=.33);var r,o=Math.floor(n/s()(e).width());i&&o>8&&(o=8),!i&&o<8&&(o=8),e.length-1-o>=1&&o--,s()("#more-apps a").removeClass("active");for(var a=0;a<e.length-1;a++){var l=s()(e[a]).data("id");a<o?(s()(e[a]).removeClass("hidden"),s()("#apps li[data-id="+l+"]").addClass("in-header"),r=e[a]):(s()(e[a]).addClass("hidden"),s()("#apps li[data-id="+l+"]").removeClass("in-header"),o>0&&s()(e[a]).children("a").hasClass("active")&&(s()(r).addClass("hidden"),s()("#apps li[data-id="+s()(r).data("id")+"]").removeClass("in-header"),s()(e[a]).removeClass("hidden"),s()("#apps li[data-id="+l+"]").addClass("in-header")))}0===s()("#apps li:not(.in-header)").length?(s()("#more-apps").hide(),s()("#navigation").hide()):s()("#more-apps").show()}

Well, we can still patch that, can we?

Continue reading "Fixing the Nextcloud menu to show more than eight application icons"