From e766629d20df9c4b1d58e77048162822c7cbcb87 Mon Sep 17 00:00:00 2001 From: Jeff Date: Thu, 17 Aug 2023 16:17:15 -0400 Subject: [PATCH] Add new article --- www/basic.css | 103 ++++++++++++++++++ ..._time_to_rethink_the_command_line_shell.md | 86 +++++++++++++++ 2 files changed, 189 insertions(+) create mode 100644 www/basic.css create mode 100644 www/src/its_time_to_rethink_the_command_line_shell.md diff --git a/www/basic.css b/www/basic.css new file mode 100644 index 0000000..335583b --- /dev/null +++ b/www/basic.css @@ -0,0 +1,103 @@ +/* Basic.css */ + +* {box-sizing: border-box} + +:root{ +--sans: 1em/1.6 system-ui, -apple-system, BlinkMacSystemFont, Segoe UI, Roboto, Oxygen, Ubuntu, Cantarell, Droid Sans, Helvetica Neue, Fira Sans, sans-serif; +--mono: SFMono-Regular, Consolas, 'Liberation Mono', Menlo, Courier, 'Courier New', monospace; +--c1:#0074d9; +--c2:#eee; +--c3:#f00; +--c4:#000; +--c5:#fff; +--m1: 8px; +--rc: 8px; +} + +@media (prefers-color-scheme: dark) { + :root { + --c2:#333; + --c3:#1e1f20; + --c4:#fff; + } +} + +html { + -ms-text-size-adjust: 100%; + -webkit-text-size-adjust: 100%; + +} + +/* General settings */ + +body { + margin: 0; + font: var(--sans); + font-weight: 400; + font-style: normal; + text-rendering: optimizeLegibility; + -webkit-font-smoothing: antialiased; + background-color: var(--c3); + color: var(--c4); +} +img, iframe {border: none; max-width: 100%} + +a {color: var(--c1); text-decoration:none} + +a:hover {color: var(--c1); text-decoration: underline} + +pre {font: 1em/1.6 var(--mono); background: var(--c2); padding: 1em; overflow: auto} + +code {font: 1em/1.6 var(--mono);} + +blockquote {border-left: 5px solid var(--c2); padding: 1em 1.5em; margin: 0} + +hr {border:0; border-bottom: 1px solid var(--c4)} + + /* Headlines */ + +h1,h2,h3,h4,h5,h6 {margin: 0.6em 0; font-weight: normal} + +h1 {font-size: 2.625em; line-height: 1.2} + +h2 {font-size: 1.625em; line-height: 1.2} + +h3 {font-size: 1.3125em; line-height: 1.24} + +h4 {font-size: 1.1875em; line-height: 1.23} + +h5,h6 {font-size: 1em; font-weight:bold} + +/* Table */ + +table {border-collapse: collapse; border-spacing: 0; margin:1em 0} + +th, td {text-align: left; vertical-align: top; border: 1px solid; padding: 0.4em} + +thead,tfoot {background: var(--c2)} + +/* Rounded Corners*/ + +pre,code,input,select,textarea,button,img {border-radius: var(--rc)} + + +/* Forms */ + +input, select, textarea {font-size: 1em; color:var(--c4); background: var(--c2); border: 0; padding: 0.6em} + +button, input[type=submit], input[type=reset], input[type="button"] { -webkit-appearance: none; font-size: 1em; display: inline-block; color: var(--c5); background: var(--c1); border: 0; margin: 4px; padding: 0.6em; cursor: pointer; text-align: center} + +button:hover, button:focus, input:hover, textarea:hover, select:hover {opacity: 0.8} + +/* Infinite Grid */ + +section {display: flex; flex-flow: row wrap} + +[style*="--c:"], section>section, aside, article {flex:var(--c,1); margin:var(--m1)} + +/* Cards */ + +article {background: var(--c2); border-radius: var(--rc); padding: 1em; box-shadow: 0px 1px 0px rgba(0,0,0,0.3)} + +[style*="--c:"]:first-child, section>section:first-child, article:first-child {margin-left:0} +[style*="--c:"]:last-child, section>section:last-child, article:last-child {margin-right:0} diff --git a/www/src/its_time_to_rethink_the_command_line_shell.md b/www/src/its_time_to_rethink_the_command_line_shell.md new file mode 100644 index 0000000..b3b1afc --- /dev/null +++ b/www/src/its_time_to_rethink_the_command_line_shell.md @@ -0,0 +1,86 @@ +# It's Time to Rethink the Command Line Shell + +I recently wrote a command-line shell and programming language called [whale]. I did so for fun and because I wanted to work out some frustrations with life on the command line by trying to build solutions. The [nushell] project was a big inspiration and assistance (I borrowed their readline library) and they were probably the first to make me wish for structured data. + +## The String Problem + +The old shells (i.e sh, bash, fish and zsh) differ in meaningful ways. For example, control flow, variable assignment, how they read startup scripts, etc. But fundamentally, they all represent data the same way: as text strings. This is a problem because, unless the user entered it, a string is rarely the most meaninful way to present data. + +String-based shells work well when you are evoking commands with flags and arguments because those are all user-entered strings. But when you fetch some JSON data from the web or export a database as CSV, you are suddenly left with an unwieldy wall of text that the shell can't handle on its own. This is not actually a design flaw, these shells were designed to solve problems related to text because they ran on systems that were almost entirely text-based. + +> All the core UNIX system tools were designed so that they could operate +> together. The original text-based editors (and even TeX and LaTeX) use ASCII +> (the American text encoding standard; an open standard) and you can use tools +> such as; sed, awk, vi, grep, cat, more, tr and various other text-based tools +> in conjunction with these editors. + +But structured data has come a long way since those shells were designed. Three of the most popular shells (sh, bash and zsh) are older than both JSON and CSV. The CSV format was formally defined in 2005, the year that fish was released. And JSON's formal definition did not come until 2017. These shells use a single ambiguous data type (the string) because it is universal and ad-hoc parsing through sed and awk was the best that could be done. + +## The Unix Tool Problem + +The whole point of a command-line shell, in my estimation, is to allow the user to pass data to and between single-purpose programs. The [unix tool] philosophy is to do one thing and do it well. One writes programs by processing input into output. This is a simple yet monumental computing and programming paradigm. + +Is it possible to disambiguate data on the command line by using more than just strings without violating this philosophy? In this paradigm, it may seem that the shell *should* only recognize data in the form of an ambiguous type like the string because the meaningful work is supposed to happen inside of programs, not between them. + +But a shell that implements structured data correctly *can* work within the Unix tools philosophy and can even compliment its features. + +## Solving a Problem with Strings + +Let's write a function in fish that will allow us to save a quick note. + +```fish +function new_note + set -l message $argv[1] + set -l time (date) + set -l note "$time | $message" + + note >> notes.txt +end + +new_note "buy milk" +new_note "take out garbage" +``` + +This will give us a file like this. + +```txt +Thu Aug 17 01:00:00 AM EDT 2023 | buy milk +Thu Aug 17 02:00:00 AM EDT 2023 | take out garbage +``` + +In order to process this file, we have to start by treating the entire thing as single string. Then a program like awk, sed or grep may divide it by line before searching and interpolating positional values. This will work as long as the strings are uniform enough to pose as structured data, but it is a feeble framework that can easily be compromised by something as simple as two programs parsing dates differently. + +## Solving the Same Problem with Structured Data + +Here's how the still-experimental shell [whale] handles the same job of defining a function that saves a note to a file. + +```whale +new_note = ' + note.message = input; + note.time = now(); + + notes = from_toml(read_file("notes.toml")); + notes:push(note); + + data = notes:to_toml(); + data:write("notes.toml"); +'; + +new_note("buy milk"); +new_note("take out garbage"); +``` + +Notice that we first declare variables with dot notation to group them. Then we read our existing notes from a TOML file into a variable. This variable is a **list**, a data structure that is a simple contiguous sequence. We can then call **push** on the list to add new item to the end. Finally, we write the file using our new list of notes. + +TOML, unlike JSON, has a time data type that makes it a good choice when saving dates and times. That solves the problem of different programs processing dates differently, `from_toml` and `to_toml`are the shell's universal means of conversion for this format. There are also conversion macros for CSV and JSON. These are infallible: they create only valid output and will convert any valid input. This improvement is virtually without downsides: any structured data can still be treated as a string and *structured data can be converted between formats*, which improves interoperability. + +```whale +toml_string = read_file("data.toml"); +parsed_data = from_toml(toml_string); +json_string = to_json(parsed_data); +``` + +## Writing Unix Tools for Structured Data + +Rust users are lucky to have [serde], a library that can create in-memory values from data formats. If you are compiling a program to be used as a command-line tool, I would urge you to think about how you might use structured data as inputs and/or outputs. [Pandoc], for example, can use a YAML file as an alternative to command-line arguments but I would prefer it to have an additional option to pass that YAML to standard input. +