Begin restructuring the docs; Write some docs

README.md

@@ -143,155 +143,6 @@ Tests are written in three places: in the Rust library, in Dust as examples and
 
 Tree Sitter generates a concrete syntax tree, which Dust traverses to create an abstract syntax tree that can run the Dust code. The CST generation is an extra step but it allows easy testing of the parser, defining the language in one file and makes the syntax easy to modify and expand. Because it uses Tree Sitter, developer-friendly features like syntax highlighting and code navigation are already available in any text editor that supports Tree Sitter.
 
-## The Dust Programming Language
-
-Dust is easy to learn. Aside from this guide, the best way to learn Dust is to read the examples and tests to get a better idea of what it can do.
-
-### Declaring Variables
-
-Variables have two parts: a key and a value. The key is always a string. The value can be any of the following data types:
-
-- string
-- integer
-- float
-- boolean
-- list
-- map
-- option
-- function
-
-Here are some examples of variables in dust.
-
-```dust
-string = "foobar"
-integer = 42
-float = 42.42
-list = [1 2 string integer float] # Commas are optional when writing lists.
-map = {
-    key = 'value'
-}
-```
-
-Note that strings can be wrapped with any kind of quote: single, double or backticks. Numbers are always integers by default. Floats are declared by adding a decimal. If you divide integers or do any kind of math with a float, you will create a float value.
-
-Dust enforces strict type checking, but you don't usually need to write the type, dust can figure it out on its own. The **number** and **any** types are special types that allow you to relax the type bounds.
-
-```dust
-string <str> = "foobar"
-integer <int> = 42
-float <float> = 42.42
-
-numbers <[number]> = [integer float]
-
-stuff <[any]> = [string integer float]
-```
-
-### Lists
-
-Lists are sequential collections. They can be built by grouping values with square brackets. Commas are optional. Values can be indexed by their position using a colon `:` followed by an integer. Dust lists are zero-indexed.
-
-```dust
-list = [true 41 "Ok"]
-
-assert_equal(list:0 true)
-
-the_answer = list:1 + 1
-
-assert_equal(the_answer, 42) # You can use commas when passing values a function. 
-```
-
-### Maps
-
-Maps are flexible collections with arbitrary key-value pairs, similar to JSON objects. A map is created with a pair of curly braces and its entries are variables declared inside those braces. Map contents can be accessed using a colon `:`.
-
-```dust
-reminder = {
-    message = "Buy milk"
-    tags = ["groceries", "home"]
-}
-
-output(reminder:message)
-```
-
-### Loops
-
-A **while** loop continues until a predicate is false.
-
-```dust
-i = 0
-while i < 10 {
-    output(i)
-    i += 1
-}
-```
-
-A **for** loop operates on a list without mutating it or the items inside. It does not return a value.
-
-```dust
-list = [ 1, 2, 3 ]
-
-for number in list {
-    output(number + 1)
-}
-```
-
-### Functions
-
-Functions are first-class values in dust, so they are assigned to variables like any other value.
-
-```dust
-# This simple function has no arguments and no return value.
-say_hi = () <none> {
-    output("hi")
-}
-
-# This function has one argument and will return a value.
-add_one = (number <num>) <num> {
-    number + 1
-}
-
-say_hi()
-assert_equal(add_one(3), 4)
-```
-
-You don't need commas when listing arguments and you don't need to add whitespace inside the function body but doing so may make your code easier to read.
-
-### Option
-
-An **option** represents a value that may not be present. It has two variants: **some** and **none**. Dust includes built-in functions to work with option values: `is_none`, `is_some` and `either_or`.
-
-```dust
-say_something = (message <option(str)>) <str> {
-    either_or(message, "hiya")
-}
-
-say_something(some("goodbye"))
-# goodbye
-
-say_something(none)
-# hiya
-```
-
-### Concurrency
-
-Dust features effortless concurrency anywhere in your code. Any block of code can be made to run its contents asynchronously. Dust's concurrency is written in safe Rust and uses a thread pool whose size depends on the number of cores available.
-
-```dust
-# An async block will run each statement in its own thread.
-async {
-    output(random_integer())
-    output(random_float())
-    output(random_boolean())
-}
-```
-
-```dust
-data = async {
-    output("Reading a file...")
-    read("examples/assets/faithful.csv")
-}
-```
-
 ## Acknowledgements
 
 Dust began as a fork of [evalexpr]. Some of the original code is still in place but the project has dramatically changed and no longer uses any of its parsing or interpreting.

docs/language.md

@@ -0,0 +1,216 @@
+# The Dust Programming Language
+
+<!--toc:start-->
+- [The Dust Programming Language](#the-dust-programming-language)
+  - [Value](#value)
+    - [Boolean](#boolean)
+    - [Integer](#integer)
+    - [Float](#float)
+    - [String](#string)
+    - [List](#list)
+    - [Maps](#maps)
+    - [Function](#function)
+    - [Option](#option)
+  - [Types](#types)
+  - [Loops](#loops)
+    - [While](#while)
+    - [For/Async For](#forasync-for)
+  - [Concurrency](#concurrency)
+<!--toc:end-->
+
+Dust is a general purpose, interpreted and strictly typed language with first-class functions. This guide is an in-depth description of the abstractions and concepts that are used to implement the language.
+
+Dust aims to be panic-free. That means that the interpreter will only fail to run a program due to and intended error, such as a type error or a syntax error.
+
+## Values
+
+There are ten kinds of value in Dust. Some are very simple and are parsed directly from the source code, some are collections and others are used in special ways, like functions and structures. All values can be assinged to an [identifier][].
+
+### Boolean
+
+Booleans are true or false. They are represented by the literal tokens `true` and `false`.
+
+### Integer
+
+Integers are whole numbers that may be positive, negative or zero. Internally, each integer is a signed 64-bit value. Integers always **overflow** when their maximum or minimum value is reached. Overflowing means that if the value is too high or low for the 64-bit integer, it will wrap around. So `maximum_value + 1` yields the minimum value and `minimum_value - 1` yields the maximum value.
+
+```dust
+42
+```
+
+### Float
+
+A float is a numeric value with a decimal. Floats are 64-bit and, like integers, will **overflow** at their bounds.
+
+```dust
+42.0
+```
+
+### String
+
+A string is a **utf-8** sequence used to represent text. Strings can be wrapped in single or double quotes as well as backticks.
+
+```dust
+'42'
+"42"
+`42`
+'forty-two'
+```
+
+### List
+
+A list is **collection** of values stored as a sequence and accessible by indexing their position with an integer. Lists indexes begin at zero for the first item.
+
+```dust
+[ 42 'forty-two' ]
+[ 123, 'one', 'two', 'three' ]
+```
+
+Note that the commas are optional, including trailing commas.
+
+```dust
+[1 2 3 4 5]:2
+# Output: 3
+```
+
+### Maps
+
+Maps are flexible collections with arbitrary **key-value pairs**, similar to JSON objects. A map is created with a pair of curly braces and its entries are variables declared inside those braces. Map contents can be accessed using a colon `:`. Commas may optionally be included after the key-value pairs.
+
+```dust
+reminder = {
+    message = "Buy milk"
+    tags = ["groceries", "home"]
+}
+
+reminder:message
+# Output: Buy milk
+```
+
+Internally a map is represented by a b-tree. The implicit advantage of using a b-tree instead of a hash map is that a b-tree is sorted and therefore can be easily compared to another. Maps are also used by the interpreter as the data structure for a **[context][]**.
+
+The map stores an [identifier][]'s key, the value it represents and the value's type. For internal use by the interpreter, a type can be set to a key without a value. This makes it possible to check the types of values before they are computed.
+
+### Function
+
+Functions are first-class values in Dust, so they are assigned to variables like any other value.
+
+```dust
+# This simple function has no arguments and no return value.
+say_hi = () <none> {
+    output("hi") # The "output" function is a built-in that prints to stdout.
+}
+
+# This function has one argument and will return a value.
+add_one = (number <num>) <num> {
+    number + 1
+}
+
+say_hi()
+assert_equal(add_one(3), 4)
+```
+
+You don't need commas when listing arguments and you don't need to add whitespace inside the function body but doing so may make your code easier to read.
+
+### Option
+
+An option represents a value that may not be present. It has two variants: **some** and **none**. 
+
+```dust
+say_something = (message <option(str)>) <str> {
+    either_or(message, "hiya")
+}
+
+say_something(some("goodbye"))
+# goodbye
+
+say_something(none)
+# hiya
+```
+
+Dust includes built-in functions to work with option values: `is_none`, `is_some` and `either_or`.
+
+### Structure
+
+A structure is an **concrete type value**. It is a value, like any other, and can be [assigned](#assignment) to an [identifier](). It can also be instantiated as a [map]() that will only allow the variables present in the structure. Default values may be provided for each variable in the structure, which will be propagated to the map it creates. Values without defaults must be given a value during instantiation.
+
+```dust
+struct User {
+    name <str>
+    email <str>
+    id <int> = generate_id()
+}
+
+bob = new User {
+    name = "Bob"
+    email = "bob@example.com"
+}
+
+# The variable "bob" is a structured map.
+```
+
+A map created by using [new]() is called a **structured map**. In other languages it may be called a "homomorphic mapped type". Dust will generate errors if you try to set any values on the structured map that are not allowed by the structure.
+
+## Types
+
+Dust enforces strict type checking, but you don't usually need to write the type, dust can figure it out on its own. The **number** and **any** types are special types that allow you to relax the type bounds.
+
+```dust
+string <str> = "foobar"
+integer <int> = 42
+float <float> = 42.42
+
+numbers <[number]> = [integer float]
+
+stuff <[any]> = [string integer float]
+```
+
+## Identifiers
+
+## Assignment
+
+## Loops
+
+### While
+
+A **while** loop continues until a predicate is false.
+
+```dust
+i = 0
+while i < 10 {
+    output(i)
+    i += 1
+}
+```
+
+### For/Async For
+
+A **for** loop operates on a list without mutating it or the items inside. It does not return a value.
+
+```dust
+list = [ 1, 2, 3 ]
+
+for number in list {
+    output(number + 1)
+}
+```
+
+## Concurrency
+
+Dust features effortless concurrency anywhere in your code. Any block of code can be made to run its contents asynchronously. Dust's concurrency is written in safe Rust and uses a thread pool whose size depends on the number of cores available.
+
+```dust
+# An async block will run each statement in its own thread.
+async {
+    output(random_integer())
+    output(random_float())
+    output(random_boolean())
+}
+```
+
+```dust
+data = async {
+    output("Reading a file...")
+    read("examples/assets/faithful.csv")
+}
+```