A piggy bank of commands, fixes, succinct reviews, some mini articles and technical opinions from a (mostly) Perl developer.

Jump to

Quick reference

Command-line data processing 2023

Some developer tools with CLI  for processing XML, XHTML, HTML, JSON, YAML, etc.

XML

  • xsh (perl - Choroba)
    • cpm XML::XSH2
    • xsh -P file.xml
    • ls
    • help ls
    • help | less
    • <TAB> autocompletion
  • xmllint
    • xmllint --xpath "//foo" file.xml
    • xmllint --shell file.xml
  • xmltarlet
  • xq (golang - )
    • apt-get install xq
  • xq (python - jeffbr13)
    • pip install xq

JSON

  • jq
    • cat file.json | jq . # format
    • cat file.json | jq '.[]' # extract array

Search

  • fzf
  • ripgrep
  • ag
  • ack
    • Doesn't search "binary" files by default
  • vim - for searching files

Windows 10 robocopy basics

robocopy c:\temp\source c:\temp\destination /E /DCOPY:DAT /R:10 /W:3

Running Emby on Linux

wget https://github.com/MediaBrowser/Emby.Releases/releases/download/4.5.4.0/emby-server-deb_4.5.4.0_amd64.deb


sudo dpkg -i emby-server-deb_4.5.4.0_amd64.deb


sudo systemctl status emby-server.service


How to maintain software

You have a massive legacy spaghetti mess of code, and you need to add a new feature.

Write tests first.

Put all your new code in a new module, and only insert single lines into the legacy code to call the new code. This allows you to properly encapsulate the new code, wrap it in try/catch blocks, etc. and not break the legacy system.

Pass whole objects into the new code, this is a necessary consequence of encapsulation.


How big is libgen? Answer: 131TiB (= 144Tb)

# First method: 131TiB

$ wget https://phillm.net/libgen-stats.php -O libgen-stats.json


$ cat libgen-stats.json | jq '.[]|.size_bytes' | perl -lne'$x+=$_;END{print$x}'

144116297318150


$ bc -l

144116297318150 / 1024 / 1024 / 1024 / 1024

131.07300884998949186410 Tebibytes


$ bc -l

144116297318150 / 1000 / 1000 / 1000 / 1000

144.11629731815000000000 Terabytes