Write docs; Add fs built-in functions

README.md

@@ -1,5 +1,7 @@
 # Dust
 
+!!! Dust is an experimental project under active development. !!!
+
 Dust is a general purpose programming language that emphasises concurrency and correctness.
 
 A basic dust program:
@@ -17,14 +19,28 @@ async {
 }
 ```
 
-You can make *any* block, i.e. `{}`, run its statements in parallel by changing it to `async {}`.
+You can use Dust to run complex operations simply and safely, you can even invoke other programs, run them at the same time, capture their output, and pipe them together.
 
 ```dust
-if random_boolean() {
-    output("Do something...")
-} else async {
-    output("Do something else instead...")
-    output("And another thing at the same time...")
+# Run each statment in this block in its own thread.
+async { 
+    # Invoke another program and capture its output.
+    ip_info = ^ip address;
+
+    # Pipe the output to another program.
+    ^ls -1 --all --long docs/ | ^rg .md | ^echo;
+
+    # This block is not async and the statements will be run in order.
+    {
+        file = fs:read_file('Cargo.toml')
+
+        for line in str:lines(file) {
+            if str:contains(line, 'author') {
+                output(line)
+                break
+            }
+        }
+    }
 }
 ```
 
@@ -49,15 +65,24 @@ Dust is an interpreted, strictly typed language with first class functions. It e
 - Safety: Written in safe, stable Rust.
 - Correctness: Type checking makes it easy to write good code.
 
+## Installation
+
+You must have the default rust toolchain installed and up-to-date. Install [rustup] if it is not already installed. Run `cargo install dust-lang` then run `dust` to start the interactive shell.
+
+To build from source, clone the repository and build the parser. To do so, enter the `tree-sitter-dust` directory and run `tree-sitter-generate`. In the project root, run `cargo run` to start the shell. To see other command line options, use `cargo run -- --help`.
+
 ## Usage
 
-Dust is an experimental project under active development. At this stage, features come and go and the API is always changing. It should not be considered for serious use yet.
+After installation, the command line interpreter can be given source code to run or it can launch the command-line shell.
 
 ```sh
 cargo install dust-lang
 dust -c "output('Hello world!')"
+# Output: Hello world!
 ```
 
+Run `dust --help` to see the available commands and options.
+
 ```txt
 General purpose programming language
 
@@ -77,17 +102,12 @@ Options:
 
 ## Dust Language
 
-See [Language Reference](/docs/language.md) for more information.
-
-## Installation
-
-You must have the default rust toolchain installed and up-to-date. Install [rustup] if it is not already installed. Run `cargo install dust-lang` then run `dust` to start the interactive shell. Use `dust --help` to see the full command line options.
-
-To build from source, clone the repository and build the parser. To do so, enter the `tree-sitter-dust` directory and run `tree-sitter-generate`. In the project root, run `cargo run` to start the shell. To see other command line options, use `cargo run -- --help`.
+See the [Language Reference](/docs/language.md) for more information.
 
 ## Benchmarks
 
-Dust is at a very early development stage but performs strongly in preliminary benchmarks. The examples given were tested using [Hyperfine] on a single-core cloud instance with 1024 MB RAM. Each test was run 1000 times. The test script is shown below. Each test asks the program to read a JSON file and count the objects. Dust is a command line shell, programming language and data manipulation tool so three appropriate targets were chosen for comparison: nushell, NodeJS and jq. The programs produced identical output with the exception that NodeJS printed in color.
+Dust is at an early development stage and these tests are overly simple. Better benchmarks are needed to get a realistic idea of how Dust performs real work. For now, these tests are just for fun.
+The examples given were tested using [Hyperfine] on a single-core cloud instance with 1024 MB RAM. Each test was run 1000 times. The test script is shown below. Each test asks the program to read a JSON file and count the objects. Dust is a command line shell, programming language and data manipulation tool so three appropriate targets were chosen for comparison: nushell, NodeJS and jq. The programs produced identical output with the exception that NodeJS printed in color.
 
 For the first test, a file with four entries was used.
 

src/abstract_tree/built_in_value.rs

@@ -4,7 +4,9 @@ use enum_iterator::{all, Sequence};
 use serde::{Deserialize, Serialize};
 
 use crate::{
-    built_in_functions::{json::json_functions, string::string_functions, Callable},
+    built_in_functions::{
+        fs::fs_functions, json::json_functions, string::string_functions, Callable,
+    },
     AbstractTree, BuiltInFunction, Format, Function, List, Map, Result, SyntaxNode, Type, Value,
 };
 
@@ -81,16 +83,18 @@ impl BuiltInValue {
                 &Value::Function(Function::BuiltIn(BuiltInFunction::AssertEqual))
             }
             BuiltInValue::Fs => FS.get_or_init(|| {
-                let fs_context = Map::new();
+                let mut fs_context = BTreeMap::new();
 
-                fs_context
-                    .set(
-                        "read".to_string(),
-                        Value::Function(Function::BuiltIn(BuiltInFunction::FsRead)),
-                    )
-                    .unwrap();
+                for fs_function in fs_functions() {
+                    let key = fs_function.name().to_string();
+                    let value =
+                        Value::Function(Function::BuiltIn(BuiltInFunction::Fs(fs_function)));
+                    let r#type = value.r#type();
 
-                Value::Map(fs_context)
+                    fs_context.insert(key, (value, r#type));
+                }
+
+                Value::Map(Map::with_variables(fs_context))
             }),
             BuiltInValue::Json => JSON.get_or_init(|| {
                 let mut json_context = BTreeMap::new();

src/built_in_functions/fs.rs

@@ -0,0 +1,50 @@
+use std::fs::read_to_string;
+
+use enum_iterator::{all, Sequence};
+use serde::{Deserialize, Serialize};
+
+use crate::{Error, Map, Result, Type, Value};
+
+use super::Callable;
+
+pub fn fs_functions() -> impl Iterator<Item = Fs> {
+    all()
+}
+
+#[derive(Sequence, Debug, Copy, Clone, Serialize, Deserialize, PartialEq, Eq, PartialOrd, Ord)]
+pub enum Fs {
+    ReadFile,
+}
+
+impl Callable for Fs {
+    fn name(&self) -> &'static str {
+        match self {
+            Fs::ReadFile => "read_file",
+        }
+    }
+
+    fn description(&self) -> &'static str {
+        match self {
+            Fs::ReadFile => "Read the contents of a file to a string.",
+        }
+    }
+
+    fn r#type(&self) -> Type {
+        match self {
+            Fs::ReadFile => Type::function(vec![Type::String], Type::String),
+        }
+    }
+
+    fn call(&self, arguments: &[Value], _source: &str, _outer_context: &Map) -> Result<Value> {
+        match self {
+            Fs::ReadFile => {
+                Error::expect_argument_amount(self.name(), 1, arguments.len())?;
+
+                let path = arguments.first().unwrap().as_string()?;
+                let file_content = read_to_string(path.as_str())?;
+
+                Ok(Value::string(file_content))
+            }
+        }
+    }
+}

src/built_in_functions/mod.rs

@@ -1,17 +1,15 @@
+pub mod fs;
 pub mod json;
 pub mod string;
 
-use std::{
-    fmt::{self, Display, Formatter},
-    fs::read_to_string,
-};
+use std::fmt::{self, Display, Formatter};
 
 use rand::{random, thread_rng, Rng};
 use serde::{Deserialize, Serialize};
 
 use crate::{Error, Format, Map, Result, Type, Value};
 
-use self::{json::Json, string::StringFunction};
+use self::{fs::Fs, json::Json, string::StringFunction};
 
 pub trait Callable {
     fn name(&self) -> &'static str;
@@ -23,7 +21,7 @@ pub trait Callable {
 #[derive(Debug, Copy, Clone, Serialize, Deserialize, PartialEq, Eq, PartialOrd, Ord)]
 pub enum BuiltInFunction {
     AssertEqual,
-    FsRead,
+    Fs(Fs),
     Json(Json),
     Length,
     Output,
@@ -38,7 +36,7 @@ impl Callable for BuiltInFunction {
     fn name(&self) -> &'static str {
         match self {
             BuiltInFunction::AssertEqual => "assert_equal",
-            BuiltInFunction::FsRead => "read",
+            BuiltInFunction::Fs(fs_function) => fs_function.name(),
             BuiltInFunction::Json(json_function) => json_function.name(),
             BuiltInFunction::Length => "length",
             BuiltInFunction::Output => "output",
@@ -53,22 +51,22 @@ impl Callable for BuiltInFunction {
     fn description(&self) -> &'static str {
         match self {
             BuiltInFunction::AssertEqual => "assert_equal",
-            BuiltInFunction::FsRead => "read",
-            BuiltInFunction::Json(json_function) => json_function.name(),
+            BuiltInFunction::Fs(fs_function) => fs_function.description(),
+            BuiltInFunction::Json(json_function) => json_function.description(),
             BuiltInFunction::Length => "length",
             BuiltInFunction::Output => "output",
             BuiltInFunction::RandomBoolean => "boolean",
             BuiltInFunction::RandomFloat => "float",
             BuiltInFunction::RandomFrom => "from",
             BuiltInFunction::RandomInteger => "integer",
-            BuiltInFunction::String(string_function) => string_function.name(),
+            BuiltInFunction::String(string_function) => string_function.description(),
         }
     }
 
     fn r#type(&self) -> Type {
         match self {
             BuiltInFunction::AssertEqual => Type::function(vec![Type::Any, Type::Any], Type::None),
-            BuiltInFunction::FsRead => Type::function(vec![Type::String], Type::String),
+            BuiltInFunction::Fs(fs_function) => fs_function.r#type(),
             BuiltInFunction::Json(json_function) => json_function.r#type(),
             BuiltInFunction::Length => Type::function(vec![Type::Collection], Type::Integer),
             BuiltInFunction::Output => Type::function(vec![Type::Any], Type::None),
@@ -90,13 +88,8 @@ impl Callable for BuiltInFunction {
 
                 Ok(Value::Boolean(left == right))
             }
-            BuiltInFunction::FsRead => {
-                Error::expect_argument_amount(self.name(), 1, arguments.len())?;
-
-                let path = arguments.first().unwrap().as_string()?;
-                let file_content = read_to_string(path.as_str())?;
-
-                Ok(Value::string(file_content))
+            BuiltInFunction::Fs(fs_function) => {
+                fs_function.call(arguments, _source, _outer_context)
             }
             BuiltInFunction::Json(json_function) => {
                 json_function.call(arguments, _source, _outer_context)