1
0
dust/README.md

244 lines
7.8 KiB
Markdown
Raw Normal View History

2023-08-22 15:40:50 +00:00
# Dust
2023-08-23 04:38:51 +00:00
Dust is a data-oriented programming language and interactive shell. Dust can be used as a replacement for a traditional command line shell, as a scripting language and as a tool create or manage data. Dust is expression-based, has first-class functions, lexical scope and lightweight syntax.
2023-08-22 15:40:50 +00:00
A basic dust program:
```dust
output "Hello world!"
```
2023-08-23 04:38:51 +00:00
Dust can do two (or more) things at the same time with effortless concurrency:
2023-08-22 15:40:50 +00:00
```dust
2023-08-23 04:38:51 +00:00
run(
2023-08-22 15:40:50 +00:00
'output "will this one finish first?"',
'output "or will this one?"'
)
```
2023-08-23 04:38:51 +00:00
Dust can do amazing things with data. To load CSV data, isolate a column and render it as a line plot in a GUI window:
2023-08-22 15:40:50 +00:00
```dust
read("examples/assets/faithful.csv")
-> from_csv(input)
2023-08-25 14:27:48 +00:00
-> rows(input)
2023-08-23 04:38:51 +00:00
-> transform(input, 'input.1')
2023-08-22 15:40:50 +00:00
-> plot(input)
```
2023-08-23 21:28:19 +00:00
<!--toc:start-->
- [Dust](#dust)
- [Features](#features)
- [Usage](#usage)
- [Installation](#installation)
- [The Dust Programming Language](#the-dust-programming-language)
- [Variables and Data Types](#variables-and-data-types)
- [Commands](#commands)
- [Lists](#lists)
- [Maps](#maps)
- [Tables](#tables)
- [The Yield Operator](#the-yield-operator)
- [Functions](#functions)
- [Time](#time)
<!--toc:end-->
2023-08-22 15:40:50 +00:00
## Features
2023-08-23 04:38:51 +00:00
- Data visualization: GUI (not TUI) plots, graphs and charts are available from directly within dust. No external tools are needed.
- Powerful tooling: Built-in commands reduce complex tasks to plain, simple code. You can even partition disks or install software.
- Pipelines: Like a pipe in bash, dust features the yield `->` operator.
- Format conversion: Effortlessly convert between dust and formats like JSON, CSV and TOML.
- Structured data: Dust can represent data with more than just strings. Lists, maps and tables are easy to make and manage.
- Developer tools: Dust has a complete tree sitter grammar, allowing syntax highlighting and completion in most code editors.
2023-08-22 15:40:50 +00:00
## Usage
Dust is an experimental project under active development. At this stage, features come and go and the API is always changing. It should not be considered for serious use yet.
## Installation
You must have the default rust toolchain installed and up-to-date. Clone the repository and run `cargo run` to start the interactive shell. To see other command line options, use `cargo run -- --help`.
2023-08-23 21:28:19 +00:00
## Contributing
Please submit any thoughts or suggestions for this project. To contribute a new command, see the library documentation. Implementation tests are written in dust and are run by a corresponding rust test so dust tests will be run when `cargo test` is called.
2023-08-23 04:38:51 +00:00
## The Dust Programming Language
Dust is a hard fork of [evalexpr]; a simple expression language. Dust's core language features maintain this simplicity. But it can manage large, complex sets of data and perform complicated tasks through commands. It should not take long for a new user to learn the language, especially with the assistance of the shell.
2023-08-22 15:40:50 +00:00
2023-08-23 21:28:19 +00:00
If your editor supports tree sitter, you can use [tree-sitter-dust] for syntax highlighting and completion support. Aside from this guide, the best way to learn dust is to read the examples and tests to get a better idea of what dust can do.
2023-08-22 15:40:50 +00:00
### Variables and Data Types
Variables have two parts: a key and a value. The key is always a text string. The value can be any of the following data types:
- string
- integer
- floating point value
- boolean
- list
- map
- table
- function
- time
- empty
Here are some examples of variables in dust.
2023-08-23 04:38:51 +00:00
```dust
string = "The answer is 42.";
integer = 42;
float = 42.42;
list = (1, 2, string, integer, float);
map.key = "value";
empty = ();
2023-08-22 15:40:50 +00:00
```
2023-08-23 04:38:51 +00:00
### Commands
2023-08-22 15:40:50 +00:00
2023-08-23 04:38:51 +00:00
**Commands** are dust's built-in tools. Some of them can reconfigure your whole system while others are do very little. They may accept different inputs, or none at all. commands in the `random` group can be run without input, but the `random_integer` command can optionally take two numbers as in inclusive range.
2023-08-22 15:40:50 +00:00
2023-08-23 04:38:51 +00:00
```dust
2023-08-22 15:40:50 +00:00
die_roll = random_integer(1, 6);
d20_roll = random_integer(1, 20);
coin_flip = random_boolean();
```
2023-08-23 04:38:51 +00:00
Other commands can be found by pressing TAB in the interactive shell.
2023-08-22 15:40:50 +00:00
2023-08-23 04:38:51 +00:00
```dust
2023-08-22 15:40:50 +00:00
message = "I hate dust.";
replace(message, "hate", "love");
```
### Lists
2023-08-23 04:38:51 +00:00
Lists are sequential collections. They can be built by grouping values with parentheses and separating them with commas. Values can be indexed by their position to access their contents. Lists are used to represent rows in tables and most commands take a list as an argument.
2023-08-22 15:40:50 +00:00
2023-08-23 04:38:51 +00:00
```dust
2023-08-22 15:40:50 +00:00
list = (true, 42, "Ok");
2023-08-23 04:38:51 +00:00
assert_eq(list.0, true);
2023-08-22 15:40:50 +00:00
```
### Maps
Maps are flexible collections with arbitrary key-value pairs, similar to JSON objects. Under the hood, all of dust's runtime variables are stored in a map, so, as with variables, the key is always a string.
2023-08-23 04:38:51 +00:00
```dust
2023-08-22 15:40:50 +00:00
reminder.message = "Buy milk";
reminder.tags = ("groceries", "home");
json = to_json(reminder);
append_to_file(json, "info.txt");
```
### Tables
2023-08-23 04:38:51 +00:00
Tables are strict collections, each row must have a value for each column. Empty cells must be explicitly set to an empty value.
2023-08-22 15:40:50 +00:00
2023-08-23 04:38:51 +00:00
```dust
animals = create_table (
2023-08-22 15:40:50 +00:00
("name", "species", "age"),
2023-08-23 04:38:51 +00:00
(
("rover", "cat", 14),
("spot", "snake", 9),
("bob", "giraffe", 2),
)
2023-08-22 15:40:50 +00:00
);
```
2023-08-23 04:38:51 +00:00
Querying a table is similar to SQL.
```dust
names = select(animals, "name");
youngins = where(animals, 'age < 5');
old_species = select_where(animals, "species", 'age > 5')
```
2023-08-22 15:40:50 +00:00
2023-08-23 04:38:51 +00:00
The commands `create_table` and `insert` make sure that all of the memory used to hold the rows is allocated at once, so it is good practice to group your rows together instead of using a call for each row.
```dust
insert(
animals,
2023-08-22 15:40:50 +00:00
("eliza", "ostrich", 4),
("pat", "white rhino", 7),
("jim", "walrus", 9)
);
2023-08-23 04:38:51 +00:00
assert_eq(length(animals.all), 6);
2023-08-22 15:40:50 +00:00
2023-08-23 21:28:19 +00:00
by_name = sort_by(animals, "name");
2023-08-22 15:40:50 +00:00
```
### The Yield Operator
2023-08-23 04:38:51 +00:00
Like a pipe in bash, zsh or fish, the yield operator evaluates the expression on the left and passes it as input to the expression on the right. That input is always assigned to the **`input` variable** for that context. These expressions may simply contain a value or they can call a command or function that returns a value.
2023-08-22 15:40:50 +00:00
2023-08-23 04:38:51 +00:00
```dust
2023-08-22 15:40:50 +00:00
"Hello dust!" -> output(input)
```
2023-08-23 04:38:51 +00:00
This can be useful when working on the command line but to make a script easier to read or to avoid fetching the same resource multiple times, we can also declare variables. You should use `->` and variables together to write efficient, elegant scripts.
2023-08-22 15:40:50 +00:00
2023-08-23 04:38:51 +00:00
```dust
2023-08-22 15:40:50 +00:00
json = download("https://api.sampleapis.com/futurama/characters");
from_json(json)
-> select(input, "name");
-> get(input, 4)
```
### Functions
2023-08-23 04:38:51 +00:00
Functions are first-class values in dust, so they are assigned to variables like any other value. The function body is wrapped in single parentheses. To call a function, it's just like calling a command: simply pass it an argument or use an empty set of parentheses to pass an empty value.
2023-08-22 15:40:50 +00:00
In the function bod, the **`input` variable** represents whatever value is passed to the function when called.
2023-08-23 04:38:51 +00:00
```dust
2023-08-22 15:40:50 +00:00
say_hi = 'output "hi"';
add_one = 'input + 1';
say_hi();
2023-08-23 04:38:51 +00:00
assert_eq(add_one(3), 4);
2023-08-22 15:40:50 +00:00
```
This function simply passes the input to the shell's standard output.
2023-08-23 04:38:51 +00:00
```dust
2023-08-22 15:40:50 +00:00
print = 'output(input)';
```
Because functions are stored in variables, we can use collections like maps to
organize them.
2023-08-23 04:38:51 +00:00
```dust
2023-08-22 15:40:50 +00:00
math.add = 'input.0 + input.1';
math.subtract = 'input.0 - input.1';
assert_eq(math.add(2, 2), 4);
assert_eq(math.subtract(100, 1), 99);
```
### Time
2023-08-23 04:38:51 +00:00
Dust can record, parse and convert time values. Dust can parse TOML datetime
values or can create time values using commands.
2023-08-22 15:40:50 +00:00
2023-08-23 04:38:51 +00:00
```dust
2023-08-22 15:40:50 +00:00
dob = from_toml("1979-05-27T07:32:00-08:00")
output "Date of birth = " + local(dob);
```
2023-08-23 04:38:51 +00:00
```dust
2023-08-22 15:40:50 +00:00
time = now();
2023-08-23 04:38:51 +00:00
output "Universal time is " + utc(time);
output "Local time is " + local(time);
2023-08-22 15:40:50 +00:00
```
[dnf]: https://dnf.readthedocs.io/en/latest/index.html
[evalexpr]: https://github.com/ISibboI/evalexpr