jeff/dust
1
0
Fork 0
Code

Merge branch '0.3.6-std' into 0.3.6

Browse Source
This commit is contained in:
Jeff 2023-11-28 12:07:59 -05:00
commit 6484b1b307
77 changed files with 2539 additions and 2265 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
Cargo.lock generated
View File

@ -468,7 +468,6 @@ dependencies = [
"comfy-table",
"csv",
"git2",
"json",
"rand",
"rayon",
"reqwest",
@ -909,12 +908,6 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "json"
version = "0.12.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "078e285eafdfb6c4b434e0d31e8cfcb5115b651496faca5749b88fafd4f23bfd"
[[package]]
name = "kv-log-macro"
version = "1.0.7"

10
Cargo.toml
View File

@ -2,14 +2,19 @@
name = "dust-lang"
description = "Data-Oriented Programming Language"
version = "0.3.5"
repository = "https://github.com/tree-sitter/tree-sitter-dust"
edition = "2018"
repository = "https://git.jeffa.io/jeff/dust.git"
edition = "2021"
license = "MIT"
[[bin]]
name = "dust"
path = "src/main.rs"
[profile.dev]
opt-level = 1
[profile.dev.package."*"]
opt-level = 3
[dependencies]
ansi_term = "0.12.1"
async-std = { version = "1.12.0", features = ["attributes"] }
@ -17,7 +22,6 @@ clap = { version = "4.4.4", features = ["derive"] }
comfy-table = "7.0.1"
csv = "1.2.2"
git2 = "0.18.1"
json = "0.12.4"
rand = "0.8.5"
rayon = "1.8.0"
reqwest = { version = "0.11.20", features = ["blocking", "json"] }

211
README.md
View File

@ -1,6 +1,6 @@
# Dust
Dust is a programming language and interactive shell. Dust can be used as a replacement for a traditional command line shell, as a scripting language and as a data format. Dust is fast, efficient and easy to learn.
Dust is a general purpose programming language that emphasises concurrency and correctness.
A basic dust program:
@ -17,30 +17,53 @@ async {
}
```
Dust is an interpreted, general purpose language with first class functions. It is *data-oriented*, with extensive tools to manage structured and relational data. Dust also includes built-in tooling to import and export data in a variety of formats, including JSON, TOML, YAML and CSV.
You can make *any* block, i.e. `{}`, run its statements in parallel by changing it to `async {}`.
```dust
if (random_boolean) {
(output "Do something...")
} else async {
(output "Do something else instead...")
(output "And another thing at the same time...")
}
```
Dust enforces strict type checking to make sure your code is correct. Dust does *not* have a null type.
```dust
fib = |i <int>| <int> {
if i <= 1 {
1
} else {
(fib i - 1) + (fib i - 2)
}
}
```
<!--toc:start-->
- [Dust](#dust)
- [Features](#features)
- [Usage](#usage)
- [Installation](#installation)
- [Benchmarks](#benchmarks)
- [Implementation](#implementation)
- [The Dust Programming Language](#the-dust-programming-language)
- [Declaring Variables](#declaring-variables)
- [Lists](#lists)
- [Maps](#maps)
- [Tables](#tables)
- [Loops](#loops)
- [Functions](#functions)
- [Concurrency](#concurrency)
- [Implementation](#implementation)
- [Acknowledgements](#acknowledgements)
<!--toc:end-->
## Features
- Simplicity: Dust is designed to be easy to learn.
- Speed: Dust is built on [Tree Sitter] and [Rust] to prioritize performance and correctness.
- Data format: Dust is data-oriented, making it a great language for defining data.
- Format conversion: Effortlessly convert between dust and formats like JSON, CSV and TOML.
- Structured data: Dust can represent data with more than just strings. Lists, maps and tables are easy to make and manage.
- Speed: Dust is built on [Tree Sitter] and [Rust] to prioritize performance and correctness. See [Benchmarks] below.
- Concurrency: Easily and safely write code that runs in parallel.
- Safety: Written in safe, stable Rust.
- Correctness: Type checking makes it easy to write good code that works.
## Usage
@ -58,6 +81,61 @@ You must have the default rust toolchain installed and up-to-date. Install [rust
To build from source, clone the repository and build the parser. To do so, enter the `tree-sitter-dust` directory and run `tree-sitter-generate`. In the project root, run `cargo run` to start the shell. To see other command line options, use `cargo run -- --help`.
## Benchmarks
Dust is at a very early development stage but performs strongly in preliminary benchmarks. The examples given were tested using [Hyperfine] on a single-core cloud instance with 1024 MB RAM. Each test was run 1000 times. The test script is shown below. Each test asks the program to read a JSON file and count the objects. Dust is a command line shell, programming language and data manipulation tool so three appropriate targets were chosen for comparison: nushell, NodeJS and jq. The programs produced identical output with the exception that NodeJS printed in color.
For the first test, a file with four entries was used.
| Command | Mean [ms] | Min [ms] | Max [ms]
|:---|---:|---:|---:|
| Dust | 3.1 ± 0.5 | 2.4 | 8.4 |
| jq | 33.7 ± 2.2 | 30.0 | 61.8 |
| NodeJS | 226.4 ± 13.1 | 197.6 | 346.2 |
| Nushell | 51.6 ± 3.7 | 45.4 | 104.3 |
The second set of data is from the GitHub API, it consists of 100 commits from the jq GitHub repo.
| Command | Mean [ms] | Min [ms] | Max [ms] |
|:---|---:|---:|---:|
| Dust | 6.8 ± 0.6 | 5.7 | 12.0 | 2.20 ± 0.40 |
| jq | 43.3 ± 3.6 | 37.6 | 81.6 | 13.95 ± 2.49 |
| NodeJS | 224.9 ± 12.3 | 194.8 | 298.5 |
| Nushell | 59.2 ± 5.7 | 49.7 | 125.0 | 19.11 ± 3.55 |
This data came from CERN, it is a massive file of 100,000 entries.
| Command | Mean [ms] | Min [ms] | Max [ms] |
|:---|---:|---:|---:|
| Dust | 1080.8 ± 38.7 | 975.3 | 1326.6 |
| jq | 1305.3 ± 64.3 | 1159.7 | 1925.1 |
| NodeJS | 1850.5 ± 72.5 | 1641.9 | 2395.1 |
| Nushell | 1850.5 ± 86.2 | 1625.5 | 2400.7 |
The tests were run after 5 warmup runs and the cache was cleared before each run.
```sh
hyperfine \
--shell none \
--warmup 5 \
--prepare "rm -rf /root/.cache" \
--runs 1000 \
--parameter-list data_path seaCreatures.json,jq_data.json,dielectron.json \
--export-markdown test_output.md \
"dust -c '(length (from_json input))' -p {data_path}" \
"jq 'length' {data_path}" \
"node --eval \"require('node:fs').readFile('{data_path}',(err,data)=>{console.log(JSON.parse(data).length)})\"" \
"nu -c 'open {data_path} | length'"
```
## Implementation
Dust is formally defined as a Tree Sitter grammar in the tree-sitter-dust directory. Tree sitter generates a parser, written in C, from a set of rules defined in JavaScript. Dust itself is a rust binary that calls the C parser using FFI.
Tests are written in three places: in the Rust library, in Dust as examples and in the Tree Sitter test format. Generally, features are added by implementing and testing the syntax in the tree-sitter-dust repository, then writing library tests to evaluate the new syntax. Implementation tests run the Dust files in the "examples" directory and should be used to demonstrate and verify that features work together.
Tree Sitter generates a concrete syntax tree, which Dust traverses to create an abstract syntax tree that can run the Dust code. The CST generation is an extra step but it allows easy testing of the parser, defining the language in one file and makes the syntax easy to modify and expand. Because it uses Tree Sitter, developer-friendly features like syntax highlighting and code navigation are already available in any text editor that supports Tree Sitter.
## The Dust Programming Language
Dust is easy to learn. Aside from this guide, the best way to learn Dust is to read the examples and tests to get a better idea of what it can do.
@ -72,7 +150,6 @@ Variables have two parts: a key and a value. The key is always a string. The val
- boolean
- list
- map
- table
- function
Here are some examples of variables in dust.
@ -91,14 +168,14 @@ Note that strings can be wrapped with any kind of quote: single, double or backt
### Lists
Lists are sequential collections. They can be built by grouping values with square brackets. Commas are optional. Values can be indexed by their position using dot notation with an integer. Dust lists are zero-indexed.
Lists are sequential collections. They can be built by grouping values with square brackets. Commas are optional. Values can be indexed by their position using a colon `:` followed by an integer. Dust lists are zero-indexed.
```dust
list = [true 41 "Ok"]
(assert_equal list.0 true)
(assert_equal list:0 true)
the_answer = list.1 + 1
the_answer = list:1 + 1
(assert_equal the_answer, 42) # You can also use commas when passing values to
# a function.
@ -106,7 +183,7 @@ the_answer = list.1 + 1
### Maps
Maps are flexible collections with arbitrary key-value pairs, similar to JSON objects. A map is created with a pair of curly braces and its entries are variables declared inside those braces. Map contents can be accessed using dot notation.
Maps are flexible collections with arbitrary key-value pairs, similar to JSON objects. A map is created with a pair of curly braces and its entries are variables declared inside those braces. Map contents can be accessed using a colon `:`.
```dust
reminder = {
@ -114,7 +191,7 @@ reminder = {
tags = ["groceries", "home"]
}
(output reminder.message)
(output reminder:message)
```
### Loops
@ -137,111 +214,43 @@ list = [ 1, 2, 3 ]
for number in list {
(output number + 1)
}
# The original list is left unchanged.
```
To create a new list, use a **transform** loop, which modifies the values into a new list without changing the original.
An **async for** loop will run the loop operations in parallel using a thread pool.
```dust
list = [1 2 3]
new_list = transform number in list {
number - 1
async for i in [1 2 3 4 5 6 7 8 9 0] {
(output i)
}
(output new_list)
# Output: [ 0 1 2 ]
(output list)
# Output: [ 1 2 3 ]
```
To filter out some of the values in a list, use a **filter** loop.
```dust
list = filter number in [1 2 3] {
number >= 2
}
(output list)
# Output: [ 2 3 ]
```
A **find** loop will return a single value, the first item that satisfies the predicate.
```dust
found = find number in [1 2 1] {
number != 1
}
(output found)
# Output: 2
```
### Tables
Tables are strict collections, each row must have a value for each column. If a value is "missing" it should be set to an appropriate value for that type. For example, a string can be empty and a number can be set to zero. Dust table declarations consist of a list of column names, which are identifiers enclosed in pointed braces, followed by a list of rows.
```dust
animals = table <name species age> [
["rover" "cat" 14]
["spot" "snake" 9]
["bob" "giraffe" 2]
]
```
Querying a table is similar to SQL.
```dust
names = select name from animals
youngins = select species from animals {
age <= 10
}
```
The keywords `table` and `insert` make sure that all of the memory used to hold the rows is allocated at once, so it is good practice to group your rows together instead of using a call for each row.
```dust
insert into animals [
["eliza" "ostrich" 4]
["pat" "white rhino" 7]
["jim" "walrus" 9]
]
(assert_equal 6 (length animals))
```
### Functions
Functions are first-class values in dust, so they are assigned to variables like any other value. The function body is wrapped in single parentheses. To create a function, use the "function" keyword. The function's arguments are identifiers inside of a set of pointed braces and the function body is enclosed in curly braces. To call a fuction, invoke its variable name inside a set of parentheses. You don't need commas when listing arguments and you don't need to add whitespace inside the function body but doing so may make your code easier to read.
Functions are first-class values in dust, so they are assigned to variables like any other value.
```dust
say_hi = function <> {
# This simple function has no arguments and no return type.
say_hi = || {
(output "hi")
}
add_one = function <number> {
(number + 1)
# This function has one argument and will return an integer.
add_one = |number| <int> {
number + 1
}
(say_hi)
(assert_equal (add_one 3), 4)
(assert_equal 4 (add_one 3))
```
This function simply passes the input to the shell's standard output.
```dust
print = function <input> {
(output input)
}
```
You don't need commas when listing arguments and you don't need to add whitespace inside the function body but doing so may make your code easier to read.
### Concurrency
As a language written in Rust, Dust features effortless concurrency anywhere in your code.
Dust features effortless concurrency anywhere in your code. Any block of code can be made to run its contents asynchronously. Dust's concurrency is written in safe Rust and uses a thread pool whose size depends on the number of cores available.
```dust
# An async block will run each statement in its own thread.
async {
(output (random_integer))
(output (random_float))
@ -249,7 +258,7 @@ async {
}
```
In an **async** block, each statement is run in parallel. In this case, we want to read from a file and assign the data to a variable. It doesn't matter which statement finishes first, the last statement in the block will be used as the assigned value. If one of the statements in an **async** block produces an error, the other statements will stop running if they have not already finished.
If the final statement in an async block creates a value, the block will return that value just like in a normal block.
```dust
data = async {
@ -258,16 +267,12 @@ data = async {
}
```
## Implementation
## Acknowledgements
Dust is formally defined as a Tree Sitter grammar in the tree-sitter-dust directory. Tree sitter generates a parser, written in C, from a set of rules defined in Javascript. Dust itself is a rust binary that calls the C parser using FFI.
Tests are written in three places: in the Rust library, in Dust as examples and in the Tree Sitter test format. Generally, features are added by implementing and testing the syntax in the tree-sitter-dust repository, then writing library tests to evaluate the new syntax. Implementation tests run the Dust files in the "examples" directory and should be used to demonstrate and verify that features work together.
Tree Sitter generates a concrete syntax tree, which Dust traverses to create an abstract syntax tree that can run the Dust code. The CST generation is an extra step but it allows easy testing of the parser, defining the language in one file and makes the syntax easy to modify and expand. Because it uses Tree Sitter, developer-friendly features like syntax highlighting and code navigation are already available in any text editor that supports Tree Sitter.
Dust began as a fork of [evalexpr]. Some of the original code is still in place but the project has dramatically changed and no longer uses any of its parsing or interpreting.
[Tree Sitter]: https://tree-sitter.github.io/tree-sitter/
[Rust]: https://rust-lang.org
[dnf]: https://dnf.readthedocs.io/en/latest/index.html
[evalexpr]: https://github.com/ISibboI/evalexpr
[rustup]: https://rustup.rs
[Hyperfine]: https://github.com/sharkdp/hyperfine

24
examples/async.ds
View File

@ -1,33 +1,19 @@
(output "This will print first.")
(output "This will print second.")
create_random_numbers = |count| => {
create_random_numbers = |count <int>| {
numbers = [];
while (length numbers) < count {
numbers += (random_integer)
}
(output "Made " + count + " numbers.")
}
do_a_lot = async {
async {
(create_random_numbers 1000)
(output "Made 1000 numbers")
}
do_some = async {
(create_random_numbers 100)
(output "Made 100 numbers")
}
do_a_little = async {
(create_random_numbers 100)
(create_random_numbers 10)
(output "Made 10 numbers")
}
await {
do_a_lot
do_some
do_a_little
}
(output "This will print last.")

11
examples/async_download.ds Normal file
View File

@ -0,0 +1,11 @@
async {
cast = (download "https://api.sampleapis.com/futurama/cast")
characters = (download "https://api.sampleapis.com/futurama/characters")
episodes = (download "https://api.sampleapis.com/futurama/episodes")
}
cast_len = (length (from_json cast))
characters_len = (length (from_json characters))
episodes_len = (length (from_json episodes))
(output [cast_len, characters_len, episodes_len])

71
examples/clue_solver.ds
View File

@ -1,44 +1,51 @@
rooms = ['Library' 'Kitchen']
suspects = ['White' 'Green']
weapons = ['Rope' 'Lead_Pipe']
cards = [rooms suspects weapons]
take_turn = |current_room opponent_card| => {
(remove_card opponent_card)
(make_guess current_room)
}
remove_card = |opponent_card| => {
for card_list in cards {
removed = remove card from card_list
card == opponent_card
}
if (type removed) == 'empty'
output 'Card not found.'
all_cards = {
rooms = ['Library' 'Kitchen' 'Conservatory']
suspects = ['White' 'Green' 'Scarlett']
weapons = ['Rope' 'Lead_Pipe' 'Knife']
}
make_guess = |current_room| => {
if ((length suspects) == 1)
&& ((length rooms) == 1)
&& ((length weapons) == 1)
{
is_ready_to_solve = |cards <map>| <bool> {
((length cards:suspects) == 1)
&& ((length cards:rooms) == 1)
&& ((length cards:weapons) == 1)
}
take_turn = |opponent_card <str>, current_room <str>, cards <map>| <map> {
(remove_card opponent_card cards)
(make_guess current_room cards)
cards
}
remove_card = |opponent_card <str>, cards <map>| {
cards:rooms -= opponent_card
cards:suspects -= opponent_card
cards:weapons -= opponent_card
}
make_guess = |current_room <str>, cards <map>| {
if (is_ready_to_solve cards) {
(output 'It was '
+ suspects:0
+ ' in the '
+ rooms:0
+ ' with the '
+ weapons:0
+ cards:suspects:0
+ ' in the '
+ cards:rooms:0
+ ' with the '
+ cards:weapons:0
+ '!')
} else {
(output 'I accuse '
+ (random suspects)
+ (random cards:suspects)
+ ' in the '
+ current_room
+ ' with the '
+ (random weapons)
+ '!')
+ (random cards:weapons)
+ '.')
}
}
(make_guess 'Library')
(take_turn 'Rope' 'Kitchen'
(take_turn 'Library' 'Kitchen'
(take_turn 'Conservatory' 'Kitchen'
(take_turn 'White' 'Kitchen'
(take_turn 'Green' 'Kitchen'
(take_turn 'Knife' 'Kitchen' all_cards))))))

9
examples/download.ds Normal file
View File

@ -0,0 +1,9 @@
cast = (download "https://api.sampleapis.com/futurama/cast")
characters = (download "https://api.sampleapis.com/futurama/characters")
episodes = (download "https://api.sampleapis.com/futurama/episodes")
cast_len = (length (from_json cast))
characters_len = (length (from_json characters))
episodes_len = (length (from_json episodes))
(output [cast_len, characters_len, episodes_len])

21
examples/fannkuch-redux.ds
View File

@ -1,21 +0,0 @@
numbers = (from_json input)
flip_count = 0
checksum = 0
while numbers.0 != 1 {
(reverse numbers 0 numbers.0)
if flip_count % 2 == 0 {
checksum += flip_count
} else {
checksum -= flip_count
}
checksum += flip_count * 1
flip_count += 1
}
(output numbers)
(output flip_count)
(output checksum)

8
examples/fetch.ds
View File

@ -1,8 +1,10 @@
raw_data = (download "https://api.sampleapis.com/futurama/cast")
cast_data = (from_json raw_data)
names = transform cast_member in cast_data {
cast_member.name
names = []
for cast_member in cast_data {
names += cast_member:name
}
(assert_equal "Billy West", names.0)
(assert_equal "Billy West", names:0)

2
examples/fibonacci.ds
View File

@ -1,4 +1,4 @@
fib = |i| => {
fib = |i <int>| <int> {
if i <= 1 {
1
} else {

3
examples/filter_loop.ds
View File

@ -1,3 +0,0 @@
filter item in (from_json (read 'examples/assets/jq_data.json')) {
(length item.commit.committer.name) <= 10
}

7
examples/find_loop.ds
View File

@ -1,7 +0,0 @@
list = [1 2 1 3]
found = find i in list {
i == 3
}
(assert_equal 3 found)

8
examples/fizzbuzz.ds
View File

@ -5,13 +5,13 @@ while count <= 15 {
divides_by_5 = count % 5 == 0
if divides_by_3 && divides_by_5 {
output 'fizzbuzz'
(output 'fizzbuzz')
} else if divides_by_3 {
output 'fizz'
(output 'fizz')
} else if divides_by_5 {
output 'buzz'
(output 'buzz')
} else {
output count
(output count)
}
count += 1

8
examples/jq_data.ds
View File

@ -1,8 +1,12 @@
data = (from_json (read 'examples/assets/jq_data.json'))
transform commit_data in data {
{
new_data = [];
for commit_data in data {
new_data += {
message = commit_data.commit.message
name = commit_data.commit.committer.name
}
}
new_data

6
examples/list.ds
View File

@ -1,7 +1,7 @@
numbers = [1, 2, 3]
x = numbers.{0}
y = numbers.{1}
z = numbers.{2}
x = numbers:0
y = numbers:1
z = numbers:2
(assert_equal x + y, z)

4
examples/map.ds
View File

@ -5,7 +5,7 @@ dictionary = {
(output
'Dust is '
+ dictionary.dust
+ dictionary:dust
+ '! The answer is '
+ dictionary.answer
+ dictionary:answer
)

8
examples/remove_loop.ds
View File

@ -1,8 +0,0 @@
list = [1 2 1 3]
removed = remove i from list {
i == 3
}
(assert_equal 3 removed)
(assert_equal [1 2 1] list)

8
examples/sea_creatures.ds
View File

@ -8,11 +8,11 @@ data = {
}
for creature in sea_creatures {
data.creatures += creature.name
data.total_clams += creature.clams
data:creatures += creature:name
data:total_clams += creature:clams
if creature.type == 'dolphin' {
data.dolphin_clams += creature.clams
if creature:type == 'dolphin' {
data:dolphin_clams += creature:clams
}
}

4
examples/table.ds
View File

@ -10,11 +10,11 @@ test_table = table |text bool| [
["a", true]
]
test_select = select |text bool| from my_table
test_select = select |text bool| from my_table;
(assert_equal test_select, test_table)
test_table = table |text number bool| [
test_table = table |number bool| [
[1, true]
[3, true]
]

5
examples/transform_loop.ds
View File

@ -1,5 +0,0 @@
data = (from_json (read "examples/assets/jq_data.json"))
transform item in data {
item:commit:committer:name
}

15
examples/yield.ds Normal file
View File

@ -0,0 +1,15 @@
1 -> (output)
add_one = |numbers <list>| <list> {
new_numbers = []
for number in numbers {
new_numbers += number + 1
}
new_numbers
}
foo = [1, 2, 3] -> (add_one)
(assert_equal [2 3 4] foo)

40
src/abstract_tree/assignment.rs
View File

@ -1,11 +1,12 @@
use serde::{Deserialize, Serialize};
use tree_sitter::Node;
use crate::{AbstractTree, Error, Identifier, Map, Result, Statement, Value};
use crate::{AbstractTree, Error, Identifier, Map, Result, Statement, Type, Value};
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub struct Assignment {
identifier: Identifier,
r#type: Option<Type>,
operator: AssignmentOperator,
statement: Statement,
}
@ -19,10 +20,23 @@ pub enum AssignmentOperator {
impl AbstractTree for Assignment {
fn from_syntax_node(source: &str, node: Node) -> Result<Self> {
let identifier_node = node.child(0).unwrap();
Error::expect_syntax_node(source, "assignment", node)?;
let identifier_node = node.child_by_field_name("identifier").unwrap();
let identifier = Identifier::from_syntax_node(source, identifier_node)?;
let operator_node = node.child(1).unwrap().child(0).unwrap();
let type_node = node.child_by_field_name("type");
let r#type = if let Some(type_node) = type_node {
Some(Type::from_syntax_node(source, type_node)?)
} else {
None
};
let operator_node = node
.child_by_field_name("assignment_operator")
.unwrap()
.child(0)
.unwrap();
let operator = match operator_node.kind() {
"=" => AssignmentOperator::Equal,
"+=" => AssignmentOperator::PlusEqual,
@ -37,42 +51,46 @@ impl AbstractTree for Assignment {
}
};
let statement_node = node.child(2).unwrap();
let statement_node = node.child_by_field_name("statement").unwrap();
let statement = Statement::from_syntax_node(source, statement_node)?;
Ok(Assignment {
identifier,
r#type,
operator,
statement,
})
}
fn run(&self, source: &str, context: &mut Map) -> Result<Value> {
let key = self.identifier.inner().clone();
let key = self.identifier.inner();
let value = self.statement.run(source, context)?;
let mut context = context.variables_mut();
let new_value = match self.operator {
AssignmentOperator::PlusEqual => {
if let Some(mut previous_value) = context.get(&key).cloned() {
if let Some(mut previous_value) = context.variables()?.get(key).cloned() {
previous_value += value;
previous_value
} else {
Value::Empty
return Err(Error::VariableIdentifierNotFound(key.clone()));
}
}
AssignmentOperator::MinusEqual => {
if let Some(mut previous_value) = context.get(&key).cloned() {
if let Some(mut previous_value) = context.variables()?.get(key).cloned() {
previous_value -= value;
previous_value
} else {
Value::Empty
return Err(Error::VariableIdentifierNotFound(key.clone()));
}
}
AssignmentOperator::Equal => value,
};
context.insert(key, new_value);
if let Some(r#type) = &self.r#type {
r#type.check(&new_value)?;
}
context.variables_mut()?.insert(key.clone(), new_value);
Ok(Value::Empty)
}

72
src/abstract_tree/block.rs
View File

@ -1,40 +1,84 @@
use std::sync::RwLock;
use rayon::prelude::*;
use serde::{Deserialize, Serialize};
use tree_sitter::Node;
use crate::{AbstractTree, Result, Statement};
use crate::{AbstractTree, Error, Map, Result, Statement, Value};
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub struct Block {
is_async: bool,
statements: Vec<Statement>,
}
impl AbstractTree for Block {
fn from_syntax_node(source: &str, node: Node) -> Result<Self> {
debug_assert_eq!("block", node.kind());
Error::expect_syntax_node(source, "block", node)?;
let statement_count = node.child_count();
let first_child = node.child(0).unwrap();
let is_async = first_child.kind() == "async";
let statement_count = if is_async {
node.child_count() - 3
} else {
node.child_count() - 2
};
let mut statements = Vec::with_capacity(statement_count);
for index in 0..statement_count {
for index in 1..node.child_count() - 1 {
let child_node = node.child(index).unwrap();
if child_node.kind() == "statement" {
if child_node.is_named() {
let statement = Statement::from_syntax_node(source, child_node)?;
statements.push(statement);
}
}
Ok(Block { statements })
Ok(Block {
is_async,
statements,
})
}
fn run(&self, source: &str, context: &mut crate::Map) -> crate::Result<crate::Value> {
for statement in &self.statements[0..self.statements.len() - 1] {
statement.run(source, context)?;
fn run(&self, source: &str, context: &mut Map) -> Result<Value> {
if self.is_async {
let statements = &self.statements;
let final_result = RwLock::new(Ok(Value::Empty));
statements
.into_par_iter()
.enumerate()
.find_map_first(|(index, statement)| {
if let Statement::Return(expression) = statement {
return Some(expression.run(source, &mut context.clone()));
}
let result = statement.run(source, &mut context.clone());
if result.is_err() {
Some(result)
} else if index == statements.len() - 1 {
let _ = final_result.write().unwrap().as_mut().map(|_| result);
None
} else {
None
}
})
.unwrap_or(final_result.into_inner().unwrap())
} else {
let mut prev_result = None;
for statement in &self.statements {
if let Statement::Return(expression) = statement {
return expression.run(source, context);
}
prev_result = Some(statement.run(source, context));
}
prev_result.unwrap_or(Ok(Value::Empty))
}
let final_statement = self.statements.last().unwrap();
let final_value = final_statement.run(source, context)?;
Ok(final_value)
}
}

38
src/abstract_tree/built_in_function.rs
View File

@ -11,7 +11,7 @@ use reqwest::blocking::get;
use serde::{Deserialize, Serialize};
use tree_sitter::Node;
use crate::{AbstractTree, Error, Expression, List, Map, Result, Table, Value, ValueType};
use crate::{AbstractTree, Error, Expression, List, Map, Result, Table, Type, Value};
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub enum BuiltInFunction {
@ -19,6 +19,7 @@ pub enum BuiltInFunction {
Assert(Vec<Expression>),
AssertEqual(Vec<Expression>),
Download(Expression),
Context,
Help(Option<Expression>),
Length(Expression),
Output(Vec<Expression>),
@ -30,7 +31,7 @@ pub enum BuiltInFunction {
Append(Vec<Expression>),
Metadata(Expression),
Move(Vec<Expression>),
Read(Expression),
Read(Option<Expression>),
Remove(Expression),
Write(Vec<Expression>),
@ -93,6 +94,7 @@ impl AbstractTree for BuiltInFunction {
BuiltInFunction::AssertEqual(expressions)
}
"context" => BuiltInFunction::Context,
"download" => {
let expression_node = node.child(1).unwrap();
let expression = Expression::from_syntax_node(source, expression_node)?;
@ -153,8 +155,11 @@ impl AbstractTree for BuiltInFunction {
BuiltInFunction::Move(expressions)
}
"read" => {
let expression_node = node.child(1).unwrap();
let expression = Expression::from_syntax_node(source, expression_node)?;
let expression = if let Some(node) = node.child(1) {
Some(Expression::from_syntax_node(source, node)?)
} else {
None
};
BuiltInFunction::Read(expression)
}
@ -306,6 +311,7 @@ impl AbstractTree for BuiltInFunction {
Ok(Value::Empty)
}
BuiltInFunction::Context => Ok(Value::Map(context.clone())),
BuiltInFunction::Download(expression) => {
let value = expression.run(source, context)?;
let url = value.as_string()?;
@ -317,7 +323,7 @@ impl AbstractTree for BuiltInFunction {
let value = expression.run(source, context)?;
let length = match value {
Value::List(list) => list.items().len(),
Value::Map(map) => map.len(),
Value::Map(map) => map.variables()?.len(),
Value::Table(table) => table.len(),
Value::String(string) => string.chars().count(),
_ => {
@ -361,9 +367,9 @@ impl AbstractTree for BuiltInFunction {
BuiltInFunction::Type(expression) => {
let run_expression = expression.run(source, context);
let value_type = if let Ok(value) = run_expression {
value.value_type()
value.r#type()
} else if let Err(Error::VariableIdentifierNotFound(_)) = run_expression {
ValueType::Empty
Type::Any
} else {
return run_expression;
};
@ -405,7 +411,7 @@ impl AbstractTree for BuiltInFunction {
let metadata_output = Map::new();
{
let mut metadata_variables = metadata_output.variables_mut();
let mut metadata_variables = metadata_output.variables_mut()?;
metadata_variables.insert("type".to_string(), Value::String(file_type));
metadata_variables.insert("size".to_string(), Value::Integer(size));
@ -428,8 +434,13 @@ impl AbstractTree for BuiltInFunction {
Ok(Value::Empty)
}
BuiltInFunction::Read(expression) => {
let path_value = expression.run(source, context)?;
let path = PathBuf::from(path_value.as_string()?);
let path = if let Some(expression) = expression {
let path_value = expression.run(source, context)?;
PathBuf::from(path_value.as_string()?)
} else {
PathBuf::from(".")
};
let content = if path.is_dir() {
let dir = read_dir(&path)?;
let mut contents = Vec::new();
@ -572,11 +583,8 @@ impl AbstractTree for BuiltInFunction {
let value = expressions[0].run(source, context)?;
let list = value.as_list()?.items();
if list.len() < 2 {
return Err(Error::ExpectedMinLengthList {
minimum_len: 2,
actual_len: list.len(),
});
if list.len() == 1 {
return Ok(list.first().cloned().unwrap());
}
let range = 0..list.len();

67
src/abstract_tree/expression.rs
View File

@ -3,7 +3,7 @@ use tree_sitter::Node;
use crate::{
value_node::ValueNode, AbstractTree, BuiltInFunction, Error, Identifier, Index, Map, Result,
Sublist, Value,
Value, Yield,
};
use super::{function_call::FunctionCall, logic::Logic, math::Math};
@ -12,63 +12,58 @@ use super::{function_call::FunctionCall, logic::Logic, math::Math};
pub enum Expression {
Value(ValueNode),
Identifier(Identifier),
Sublist(Box<Sublist>),
Index(Box<Index>),
Math(Box<Math>),
Logic(Box<Logic>),
FunctionCall(FunctionCall),
FunctionCall(Box<FunctionCall>),
Tool(Box<BuiltInFunction>),
Yield(Box<Yield>),
}
impl AbstractTree for Expression {
fn from_syntax_node(source: &str, node: Node) -> Result<Self> {
debug_assert_eq!("expression", node.kind());
Error::expect_syntax_node(source, "expression", node)?;
for index in 0..node.child_count() {
let child = node.child(index).unwrap();
let expression = match child.kind() {
"value" => Expression::Value(ValueNode::from_syntax_node(source, child)?),
"identifier" => {
Expression::Identifier(Identifier::from_syntax_node(source, child)?)
}
"sublist" => {
Expression::Sublist(Box::new(Sublist::from_syntax_node(source, child)?))
}
"index" => Expression::Index(Box::new(Index::from_syntax_node(source, child)?)),
"math" => Expression::Math(Box::new(Math::from_syntax_node(source, child)?)),
"logic" => Expression::Logic(Box::new(Logic::from_syntax_node(source, child)?)),
"function_call" => {
Expression::FunctionCall(FunctionCall::from_syntax_node(source, child)?)
}
"tool" => {
Expression::Tool(Box::new(BuiltInFunction::from_syntax_node(source, child)?))
}
_ => continue,
};
let child = if node.child(0).unwrap().is_named() {
node.child(0).unwrap()
} else {
node.child(1).unwrap()
};
return Ok(expression);
}
let expression = match child.kind() {
"value" => Expression::Value(ValueNode::from_syntax_node(source, child)?),
"identifier" => Expression::Identifier(Identifier::from_syntax_node(source, child)?),
"index" => Expression::Index(Box::new(Index::from_syntax_node(source, child)?)),
"math" => Expression::Math(Box::new(Math::from_syntax_node(source, child)?)),
"logic" => Expression::Logic(Box::new(Logic::from_syntax_node(source, child)?)),
"function_call" => {
Expression::FunctionCall(Box::new(FunctionCall::from_syntax_node(source, child)?))
}
"tool" => Expression::Tool(Box::new(BuiltInFunction::from_syntax_node(source, child)?)),
"yield" => Expression::Yield(Box::new(Yield::from_syntax_node(source, child)?)),
_ => {
return Err(Error::UnexpectedSyntaxNode {
expected: "value, identifier, index, math, logic, function_call or yield",
actual: child.kind(),
location: child.start_position(),
relevant_source: source[child.byte_range()].to_string(),
})
}
};
let child = node.child(0).unwrap();
Err(Error::UnexpectedSyntaxNode {
expected: "value, identifier, sublist, index, math or function_call",
actual: child.kind(),
location: child.start_position(),
relevant_source: source[child.byte_range()].to_string(),
})
Ok(expression)
}
fn run(&self, source: &str, context: &mut Map) -> Result<Value> {
match self {
Expression::Value(value_node) => value_node.run(source, context),
Expression::Identifier(identifier) => identifier.run(source, context),
Expression::Sublist(sublist) => sublist.run(source, context),
Expression::Math(math) => math.run(source, context),
Expression::Logic(logic) => logic.run(source, context),
Expression::FunctionCall(function_call) => function_call.run(source, context),
Expression::Tool(tool) => tool.run(source, context),
Expression::Index(index) => index.run(source, context),
Expression::Yield(r#yield) => r#yield.run(source, context),
}
}
}

18
src/abstract_tree/filter.rs
View File

@ -19,13 +19,13 @@ impl AbstractTree for Filter {
None => None,
};
let item_id_node = node.child(1).unwrap();
let item_id_node = node.child_by_field_name("item_id").unwrap();
let item_id = Identifier::from_syntax_node(source, item_id_node)?;
let collection_node = node.child(3).unwrap();
let collection_node = node.child_by_field_name("collection").unwrap();
let collection = Expression::from_syntax_node(source, collection_node)?;
let predicate_node = node.child(5).unwrap();
let predicate_node = node.child_by_field_name("predicate").unwrap();
let predicate = Block::from_syntax_node(source, predicate_node)?;
Ok(Filter {
@ -45,6 +45,7 @@ impl AbstractTree for Filter {
Some(expression) => Some(expression.run(source, context)?.as_integer()? as usize),
None => None,
};
let loop_context = Map::clone_from(context)?;
values.par_iter().try_for_each(|value| {
if let Some(max) = count {
@ -53,11 +54,16 @@ impl AbstractTree for Filter {
}
}
let mut context = Map::new();
let mut iter_context = loop_context.clone();
context.variables_mut().insert(key.clone(), value.clone());
iter_context
.variables_mut()?
.insert(key.clone(), value.clone());
let should_include = self.predicate.run(source, &mut context)?.as_boolean()?;
let should_include = self
.predicate
.run(source, &mut iter_context)?
.as_boolean()?;
if should_include {
new_values.items_mut().push(value.clone());

43
src/abstract_tree/find.rs
View File

@ -1,7 +1,8 @@
use rayon::prelude::*;
use serde::{Deserialize, Serialize};
use tree_sitter::Node;
use crate::{AbstractTree, Block, Expression, Identifier, Map, Result, Value};
use crate::{AbstractTree, Block, Error, Expression, Identifier, Map, Result, Value};
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub struct Find {
@ -29,21 +30,41 @@ impl AbstractTree for Find {
}
fn run(&self, source: &str, context: &mut Map) -> Result<Value> {
let value = self.expression.run(source, context)?;
let values = value.as_list()?.items();
let expression_run = self.expression.run(source, context)?;
let list = expression_run.as_list()?.items();
let key = self.identifier.inner();
let mut context = context.clone();
for value in values.iter() {
context.variables_mut().insert(key.clone(), value.clone());
let find_result = list.par_iter().find_map_first(|value| {
let loop_context = Map::clone_from(context).unwrap();
let should_return = self.item.run(source, &mut context)?.as_boolean()?;
loop_context
.variables_mut()
.unwrap()
.insert(key.clone(), (*value).clone());
if should_return {
return Ok(value.clone());
let run_result = self.item.run(source, &mut loop_context.clone());
if let Ok(run_result_value) = run_result {
if let Ok(should_return) = run_result_value.as_boolean() {
if should_return {
Some(Ok(value.clone()))
} else {
None
}
} else {
Some(Err(Error::ExpectedBoolean {
actual: value.clone(),
}))
}
} else {
Some(run_result)
}
}
});
Ok(Value::Empty)
if let Some(result) = find_result {
result
} else {
Ok(Value::Empty)
}
}
}

37
src/abstract_tree/for.rs
View File

@ -7,19 +7,22 @@ use crate::{AbstractTree, Block, Error, Expression, Identifier, Map, Result, Val
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub struct For {
is_async: bool,
identifier: Identifier,
expression: Expression,
item: Block,
item_id: Identifier,
collection: Expression,
block: Block,
}
impl AbstractTree for For {
fn from_syntax_node(source: &str, node: Node) -> Result<Self> {
Error::expect_syntax_node(source, "for", node)?;
let for_node = node.child(0).unwrap();
let is_async = match for_node.kind() {
"for" => false,
"async for" => true,
_ => {
return Err(Error::UnexpectedSyntaxNode {
expected: "for",
expected: "for or async for",
actual: for_node.kind(),
location: for_node.start_position(),
relevant_source: source[for_node.byte_range()].to_string(),
@ -38,32 +41,36 @@ impl AbstractTree for For {
Ok(For {
is_async,
identifier,
expression,
item,
item_id: identifier,
collection: expression,
block: item,
})
}
fn run(&self, source: &str, context: &mut Map) -> Result<Value> {
let expression_run = self.expression.run(source, context)?;
let expression_run = self.collection.run(source, context)?;
let values = expression_run.as_list()?.items();
let key = self.identifier.inner();
let key = self.item_id.inner();
if self.is_async {
values.par_iter().try_for_each(|value| {
let mut iter_context = Map::new();
let mut iter_context = Map::clone_from(context)?;
iter_context
.variables_mut()
.variables_mut()?
.insert(key.clone(), value.clone());
self.item.run(source, &mut iter_context).map(|_value| ())
self.block.run(source, &mut iter_context).map(|_value| ())
})?;
} else {
for value in values.iter() {
context.variables_mut().insert(key.clone(), value.clone());
let loop_context = Map::clone_from(context)?;
self.item.run(source, context)?;
for value in values.iter() {
loop_context
.variables_mut()?
.insert(key.clone(), value.clone());
self.block.run(source, &mut loop_context.clone())?;
}
}

58
src/abstract_tree/function_call.rs
View File

@ -3,13 +3,13 @@ use tree_sitter::Node;
use crate::{AbstractTree, BuiltInFunction, Error, Map, Result, Value};
use super::{expression::Expression, identifier::Identifier};
use super::expression::Expression;
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub enum FunctionCall {
BuiltIn(Box<BuiltInFunction>),
ContextDefined {
name: Identifier,
name: Expression,
arguments: Vec<Expression>,
},
}
@ -18,14 +18,14 @@ impl AbstractTree for FunctionCall {
fn from_syntax_node(source: &str, node: Node) -> Result<Self> {
debug_assert_eq!("function_call", node.kind());
let function_node = node.child(0).unwrap();
let function_node = node.child(1).unwrap();
let mut arguments = Vec::new();
for index in 1..node.child_count() {
let child = node.child(index).unwrap();
for index in 2..node.child_count() - 1 {
let node = node.child(index).unwrap();
if child.is_named() {
let expression = Expression::from_syntax_node(source, child)?;
if node.is_named() {
let expression = Expression::from_syntax_node(source, node)?;
arguments.push(expression);
}
@ -36,12 +36,9 @@ impl AbstractTree for FunctionCall {
FunctionCall::BuiltIn(Box::new(function))
} else {
let identifier = Identifier::from_syntax_node(source, function_node)?;
let name = Expression::from_syntax_node(source, function_node)?;
FunctionCall::ContextDefined {
name: identifier,
arguments,
}
FunctionCall::ContextDefined { name, arguments }
};
Ok(function_call)
@ -54,23 +51,30 @@ impl AbstractTree for FunctionCall {
FunctionCall::ContextDefined { name, arguments } => (name, arguments),
};
let definition = if let Some(value) = context.variables().get(name.inner()) {
value.as_function().cloned()?
} else {
return Err(Error::FunctionIdentifierNotFound(name.clone()));
};
if let Some(parameters) = definition.identifiers() {
let parameter_expression_pairs = parameters.iter().zip(arguments.iter());
for (identifier, expression) in parameter_expression_pairs {
let key = identifier.clone().take_inner();
let value = expression.run(source, context)?;
function_context.variables_mut().insert(key, value);
let function = if let Expression::Identifier(identifier) = name {
if let Some(value) = context.variables()?.get(identifier.inner()) {
value.as_function().cloned()
} else {
return Err(Error::FunctionIdentifierNotFound(identifier.clone()));
}
} else {
let name_run = name.run(source, context)?;
name_run.as_function().cloned()
}?;
let mut function_context = Map::clone_from(context)?;
let parameter_expression_pairs = function.parameters().iter().zip(arguments.iter());
for ((identifier, r#type), expression) in parameter_expression_pairs {
let key = identifier.clone().take_inner();
let value = expression.run(source, context)?;
r#type.check(&value)?;
function_context.variables_mut()?.insert(key, value);
}
definition.body().run(source, &mut function_context)
function.run(source, &mut function_context)
}
}

4
src/abstract_tree/identifier.rs
View File

@ -22,7 +22,7 @@ impl Identifier {
impl AbstractTree for Identifier {
fn from_syntax_node(source: &str, node: Node) -> Result<Self> {
debug_assert_eq!("identifier", node.kind());
Error::expect_syntax_node(source, "identifier", node)?;
let identifier = &source[node.byte_range()];
@ -30,7 +30,7 @@ impl AbstractTree for Identifier {
}
fn run(&self, _source: &str, context: &mut Map) -> Result<Value> {
if let Some(value) = context.variables().get(&self.0) {
if let Some(value) = context.variables()?.get(&self.0) {
Ok(value.clone())
} else {
Err(Error::VariableIdentifierNotFound(self.inner().clone()))

79
src/abstract_tree/identifier_assignment.rs Normal file
View File

@ -0,0 +1,79 @@
use serde::{Deserialize, Serialize};
use tree_sitter::Node;
use crate::{AbstractTree, Error, Identifier, Map, Result, Statement, Value};
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub struct IdentifierAssignment {
identifier: Identifier,
operator: AssignmentOperator,
statement: Statement,
}
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub enum AssignmentOperator {
Equal,
PlusEqual,
MinusEqual,
}
impl AbstractTree for IdentifierAssignment {
fn from_syntax_node(source: &str, node: Node) -> Result<Self> {
let identifier_node = node.child(0).unwrap();
let identifier = Identifier::from_syntax_node(source, identifier_node)?;
let operator_node = node.child(1).unwrap().child(0).unwrap();
let operator = match operator_node.kind() {
"=" => AssignmentOperator::Equal,
"+=" => AssignmentOperator::PlusEqual,
"-=" => AssignmentOperator::MinusEqual,
_ => {
return Err(Error::UnexpectedSyntaxNode {
expected: "=, += or -=",
actual: operator_node.kind(),
location: operator_node.start_position(),
relevant_source: source[operator_node.byte_range()].to_string(),
})
}
};
let statement_node = node.child(2).unwrap();
let statement = Statement::from_syntax_node(source, statement_node)?;
Ok(IdentifierAssignment {
identifier,
operator,
statement,
})
}
fn run(&self, source: &str, context: &mut Map) -> Result<Value> {
let key = self.identifier.inner().clone();
let value = self.statement.run(source, context)?;
let mut context = context.variables_mut();
let new_value = match self.operator {
AssignmentOperator::PlusEqual => {
if let Some(mut previous_value) = context.get(&key).cloned() {
previous_value += value;
previous_value
} else {
Value::Empty
}
}
AssignmentOperator::MinusEqual => {
if let Some(mut previous_value) = context.get(&key).cloned() {
previous_value -= value;
previous_value
} else {
Value::Empty
}
}
AssignmentOperator::Equal => value,
};
context.insert(key, new_value);
Ok(Value::Empty)
}
}

56
src/abstract_tree/index.rs
View File

@ -1,13 +1,13 @@
use serde::{Deserialize, Serialize};
use tree_sitter::Node;
use crate::{AbstractTree, Error, Expression, List, Result, Value};
use crate::{AbstractTree, Error, Expression, List, Map, Result, Value};
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub struct Index {
collection: Expression,
index: Expression,
index_end: Option<Expression>,
pub collection: Expression,
pub index: Expression,
pub index_end: Option<Expression>,
}
impl AbstractTree for Index {
@ -32,10 +32,10 @@ impl AbstractTree for Index {
})
}
fn run(&self, source: &str, context: &mut crate::Map) -> crate::Result<crate::Value> {
let value = self.collection.run(source, context)?;
fn run(&self, source: &str, context: &mut Map) -> Result<Value> {
let collection = self.collection.run(source, context)?;
match value {
match collection {
Value::List(list) => {
let index = self.index.run(source, context)?.as_integer()? as usize;
@ -50,8 +50,17 @@ impl AbstractTree for Index {
Ok(item)
}
Value::Map(mut map) => {
let value = self.index.run(source, &mut map)?;
Value::Map(map) => {
let value = if let Expression::Identifier(identifier) = &self.index {
let key = identifier.inner();
map.variables()?.get(key).cloned().unwrap_or(Value::Empty)
} else {
let value = self.index.run(source, context)?;
let key = value.as_string()?;
map.variables()?.get(key).cloned().unwrap_or(Value::Empty)
};
Ok(value)
}
@ -61,7 +70,34 @@ impl AbstractTree for Index {
Ok(Value::String(item.to_string()))
}
_ => Err(Error::ExpectedCollection { actual: value }),
_ => Err(Error::ExpectedCollection { actual: collection }),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::evaluate;
#[test]
fn evaluate_list_index() {
let test = evaluate("x = [1 [2] 3] x:1:0").unwrap();
assert_eq!(Value::Integer(2), test);
}
#[test]
fn evaluate_map_index() {
let test = evaluate("x = {y = {z = 2}} x:y:z").unwrap();
assert_eq!(Value::Integer(2), test);
}
#[test]
fn evaluate_complex_index() {
let test = evaluate("x = [1 2 3]; y = || <int> { 0 } x:((y))").unwrap();
assert_eq!(Value::Integer(1), test);
}
}

93
src/abstract_tree/index_assignment.rs Normal file
View File

@ -0,0 +1,93 @@
use serde::{Deserialize, Serialize};
use tree_sitter::Node;
use crate::{AbstractTree, Error, Index, Map, Result, Statement, Value};
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub struct IndexAssignment {
index: Index,
operator: AssignmentOperator,
statement: Statement,
}
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub enum AssignmentOperator {
Equal,
PlusEqual,
MinusEqual,
}
impl AbstractTree for IndexAssignment {
fn from_syntax_node(source: &str, node: Node) -> Result<Self> {
Error::expect_syntax_node(source, "index_assignment", node)?;
let index_node = node.child(0).unwrap();
let index = Index::from_syntax_node(source, index_node)?;
let operator_node = node.child(1).unwrap().child(0).unwrap();
let operator = match operator_node.kind() {
"=" => AssignmentOperator::Equal,
"+=" => AssignmentOperator::PlusEqual,
"-=" => AssignmentOperator::MinusEqual,
_ => {
return Err(Error::UnexpectedSyntaxNode {
expected: "=, += or -=",
actual: operator_node.kind(),
location: operator_node.start_position(),
relevant_source: source[operator_node.byte_range()].to_string(),
})
}
};
let statement_node = node.child(2).unwrap();
let statement = Statement::from_syntax_node(source, statement_node)?;
Ok(IndexAssignment {
index,
operator,
statement,
})
}
fn run(&self, source: &str, context: &mut Map) -> Result<Value> {
let index_collection = self.index.collection.run(source, context)?;
let index_context = index_collection.as_map().unwrap_or(&context);
let index_key = if let crate::Expression::Identifier(identifier) = &self.index.index {
identifier.inner()
} else {
return Err(Error::VariableIdentifierNotFound(
self.index.index.run(source, context)?.to_string(),
));
};
let value = self.statement.run(source, &mut context.clone())?;
let new_value = match self.operator {
AssignmentOperator::PlusEqual => {
if let Some(mut previous_value) = index_context.variables()?.get(index_key).cloned()
{
previous_value += value;
previous_value
} else {
Value::Empty
}
}
AssignmentOperator::MinusEqual => {
if let Some(mut previous_value) = index_context.variables()?.get(index_key).cloned()
{
previous_value -= value;
previous_value
} else {
Value::Empty
}
}
AssignmentOperator::Equal => value,
};
index_context
.variables_mut()?
.insert(index_key.clone(), new_value);
Ok(Value::Empty)
}
}

2
src/abstract_tree/insert.rs
View File

@ -36,7 +36,7 @@ impl AbstractTree for Insert {
}
context
.variables_mut()
.variables_mut()?
.insert(table_name, Value::Table(table));
Ok(Value::Empty)

14
src/abstract_tree/mod.rs
View File

@ -7,7 +7,6 @@
//! examples.
pub mod assignment;
pub mod r#await;
pub mod block;
pub mod built_in_function;
pub mod expression;
@ -18,6 +17,7 @@ pub mod function_call;
pub mod identifier;
pub mod if_else;
pub mod index;
pub mod index_assignment;
pub mod insert;
pub mod logic;
pub mod r#match;
@ -25,16 +25,18 @@ pub mod math;
pub mod remove;
pub mod select;
pub mod statement;
pub mod sublist;
pub mod transform;
pub mod r#type;
pub mod r#use;
pub mod value_node;
pub mod r#while;
pub mod r#yield;
pub use {
assignment::*, block::*, built_in_function::*, expression::*, filter::*, find::*,
function_call::*, identifier::*, if_else::*, index::*, insert::*, logic::*, math::*,
r#await::*, r#await::*, r#for::*, r#match::*, r#while::*, remove::*, select::*, statement::*,
sublist::*, transform::*, value_node::*,
function_call::*, identifier::*, if_else::*, index::*, index_assignment::IndexAssignment,
insert::*, logic::*, math::*, r#for::*, r#match::*, r#type::*, r#use::*, r#while::*,
r#yield::*, remove::*, select::*, statement::*, transform::*, value_node::*,
};
use tree_sitter::Node;
@ -55,6 +57,6 @@ pub trait AbstractTree: Sized {
/// node's byte range.
fn from_syntax_node(source: &str, node: Node) -> Result<Self>;
/// Execute dust code by traversing the tree
/// Execute dust code by traversing the tree.
fn run(&self, source: &str, context: &mut Map) -> Result<Value>;
}

82
src/abstract_tree/remove.rs
View File

@ -1,62 +1,72 @@
use std::sync::RwLock;
use rayon::prelude::*;
use serde::{Deserialize, Serialize};
use tree_sitter::Node;
use crate::{AbstractTree, Block, Expression, Identifier, Map, Result, Value};
use crate::{AbstractTree, Block, Error, Expression, Identifier, Map, Result, Value};
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub struct Remove {
identifier: Identifier,
expression: Expression,
item: Block,
item_id: Identifier,
collection: Expression,
predicate: Block,
}
impl AbstractTree for Remove {
fn from_syntax_node(source: &str, node: Node) -> Result<Self> {
let identifier_node = node.child(1).unwrap();
let identifier = Identifier::from_syntax_node(source, identifier_node)?;
let item_id = Identifier::from_syntax_node(source, identifier_node)?;
let expression_node = node.child(3).unwrap();
let expression = Expression::from_syntax_node(source, expression_node)?;
let collection = Expression::from_syntax_node(source, expression_node)?;
let item_node = node.child(4).unwrap();
let item = Block::from_syntax_node(source, item_node)?;
let block_node = node.child(4).unwrap();
let predicate = Block::from_syntax_node(source, block_node)?;
Ok(Remove {
identifier,
expression,
item,
item_id,
collection,
predicate,
})
}
fn run(&self, source: &str, context: &mut Map) -> Result<Value> {
let expression_run = self.expression.run(source, context)?;
let values = expression_run.into_inner_list()?;
let key = self.identifier.inner();
let mut sub_context = context.clone();
let mut should_remove_index = None;
let value = self.collection.run(source, context)?;
let values = value.as_list()?;
let key = self.item_id.inner();
let should_remove_index = RwLock::new(None);
for (index, value) in values.items().iter().enumerate() {
sub_context
.variables_mut()
.insert(key.clone(), value.clone());
let should_remove = self.item.run(source, &mut sub_context)?.as_boolean()?;
if should_remove {
should_remove_index = Some(index);
match &self.expression {
Expression::Identifier(identifier) => {
sub_context
.variables_mut()
.insert(identifier.inner().clone(), Value::List(values.clone()));
}
_ => {}
values
.items()
.par_iter()
.enumerate()
.try_for_each(|(index, value)| {
if should_remove_index.read()?.is_some() {
return Ok(());
}
}
}
if let Some(index) = should_remove_index {
let iter_context = Map::clone_from(context)?;
iter_context
.variables_mut()?
.insert(key.clone(), value.clone());
let should_remove = self
.predicate
.run(source, &mut iter_context.clone())?
.as_boolean()?;
if should_remove {
let _ = should_remove_index.write()?.insert(index);
}
Ok::<(), Error>(())
})?;
let index = should_remove_index.read()?;
if let Some(index) = *index {
Ok(values.items_mut().remove(index))
} else {
Ok(Value::Empty)

24
src/abstract_tree/select.rs
View File

@ -7,7 +7,7 @@ use crate::{AbstractTree, Block, Expression, Identifier, Map, Result, Table, Val
pub struct Select {
identifiers: Vec<Identifier>,
expression: Expression,
block: Option<Block>,
predicate: Option<Block>,
}
impl AbstractTree for Select {
@ -15,6 +15,8 @@ impl AbstractTree for Select {
let mut identifiers = Vec::new();
let identifier_list = node.child(1).unwrap();
let identifier_list = node.child(1).unwrap();
for index in 1..identifier_list.child_count() - 1 {
let node = identifier_list.child(index).unwrap();
@ -27,8 +29,10 @@ impl AbstractTree for Select {
let expression_node = node.child(3).unwrap();
let expression = Expression::from_syntax_node(source, expression_node)?;
let block = if let Some(block_node) = node.child(4) {
Some(Block::from_syntax_node(source, block_node)?)
let final_node = node.child(child_count - 1).unwrap();
let predicate = if final_node.kind() == "block" {
Some(Block::from_syntax_node(source, final_node)?)
} else {
None
};
@ -36,7 +40,7 @@ impl AbstractTree for Select {
Ok(Select {
identifiers,
expression,
block,
predicate,
})
}
@ -47,7 +51,7 @@ impl AbstractTree for Select {
self.identifiers
.iter()
.cloned()
.map(|identifierier| identifierier.take_inner())
.map(|identifier| identifier.take_inner())
.collect()
} else {
old_table.headers().clone()
@ -56,13 +60,13 @@ impl AbstractTree for Select {
for row in old_table.rows() {
let mut new_row = Vec::new();
let mut row_context = Map::new();
let row_context = Map::new();
for (i, value) in row.iter().enumerate() {
let column_name = old_table.headers().get(i).unwrap();
row_context
.variables_mut()
.variables_mut()?
.insert(column_name.clone(), value.clone());
let new_table_column_index =
@ -86,8 +90,10 @@ impl AbstractTree for Select {
}
}
if let Some(where_clause) = &self.block {
let should_include = where_clause.run(source, &mut row_context)?.as_boolean()?;
if let Some(where_clause) = &self.predicate {
let should_include = where_clause
.run(source, &mut row_context.clone())?
.as_boolean()?;
if should_include {
new_table.insert(new_row)?;

35
src/abstract_tree/statement.rs
View File

@ -2,35 +2,34 @@ use serde::{Deserialize, Serialize};
use tree_sitter::Node;
use crate::{
AbstractTree, Assignment, Await, Error, Expression, Filter, Find, For, IfElse, Insert, Map,
Match, Remove, Result, Select, Transform, Value, While,
AbstractTree, Assignment, Block, Error, Expression, Filter, Find, For, IfElse, IndexAssignment,
Insert, Map, Match, Remove, Result, Select, Transform, Use, Value, While,
};
/// Abstract representation of a statement.
///
/// A statement may evaluate to an Empty value when run. If a Statement is an
/// Expression, it will always return a non-empty value when run.
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub enum Statement {
Assignment(Box<Assignment>),
Await(Await),
Return(Expression),
Expression(Expression),
IfElse(Box<IfElse>),
Match(Match),
While(Box<While>),
Async(Box<Await>),
Block(Box<Block>),
For(Box<For>),
Transform(Box<Transform>),
Filter(Box<Filter>),
Find(Box<Find>),
Remove(Box<Remove>),
Use(Use),
Select(Box<Select>),
Insert(Box<Insert>),
IndexAssignment(Box<IndexAssignment>),
}
impl AbstractTree for Statement {
fn from_syntax_node(source: &str, node: Node) -> Result<Self> {
debug_assert_eq!("statement", node.kind());
Error::expect_syntax_node(source, "statement", node)?;
let child = node.child(0).unwrap();
@ -38,7 +37,11 @@ impl AbstractTree for Statement {
"assignment" => Ok(Statement::Assignment(Box::new(
Assignment::from_syntax_node(source, child)?,
))),
"await" => Ok(Statement::Await(Await::from_syntax_node(source, child)?)),
"return" => {
let expression_node = child.child(1).unwrap();
Ok(Statement::Return(Expression::from_syntax_node(source, expression_node)?))
},
"expression" => Ok(Self::Expression(Expression::from_syntax_node(
source, child,
)?)),
@ -51,7 +54,7 @@ impl AbstractTree for Statement {
"while" => Ok(Statement::While(Box::new(While::from_syntax_node(
source, child,
)?))),
"async" => Ok(Statement::Async(Box::new(Await::from_syntax_node(
"block" => Ok(Statement::Block(Box::new(Block::from_syntax_node(
source, child,
)?))),
"for" => Ok(Statement::For(Box::new(For::from_syntax_node(
@ -72,11 +75,15 @@ impl AbstractTree for Statement {
"select" => Ok(Statement::Select(Box::new(Select::from_syntax_node(
source, child,
)?))),
"use" => Ok(Statement::Use(Use::from_syntax_node(source, child)?)),
"insert" => Ok(Statement::Insert(Box::new(Insert::from_syntax_node(
source, child,
)?))),
"index_assignment" => Ok(Statement::IndexAssignment(Box::new(IndexAssignment::from_syntax_node(
source, child,
)?))),
_ => Err(Error::UnexpectedSyntaxNode {
expected: "assignment, expression, if...else, while, for, transform, filter, tool, async, find, remove, select or insert",
expected: "assignment, expression, if...else, while, for, transform, filter, tool, async, find, remove, select, insert, index_assignment or yield",
actual: child.kind(),
location: child.start_position(),
relevant_source: source[child.byte_range()].to_string(),
@ -87,19 +94,21 @@ impl AbstractTree for Statement {
fn run(&self, source: &str, context: &mut Map) -> Result<Value> {
match self {
Statement::Assignment(assignment) => assignment.run(source, context),
Statement::Await(r#await) => r#await.run(source, context),
Statement::Return(expression) => expression.run(source, context),
Statement::Expression(expression) => expression.run(source, context),
Statement::IfElse(if_else) => if_else.run(source, context),
Statement::Match(r#match) => r#match.run(source, context),
Statement::While(r#while) => r#while.run(source, context),
Statement::Async(run) => run.run(source, context),
Statement::Block(block) => block.run(source, context),
Statement::For(r#for) => r#for.run(source, context),
Statement::Transform(transform) => transform.run(source, context),
Statement::Filter(filter) => filter.run(source, context),
Statement::Find(find) => find.run(source, context),
Statement::Remove(remove) => remove.run(source, context),
Statement::Use(run) => run.run(source, context),
Statement::Select(select) => select.run(source, context),
Statement::Insert(insert) => insert.run(source, context),
Statement::IndexAssignment(index_assignment) => index_assignment.run(source, context),
}
}
}

35
src/abstract_tree/sublist.rs
View File

@ -1,35 +0,0 @@
use serde::{Deserialize, Serialize};
use crate::{AbstractTree, Expression, List, Value};
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub struct Sublist {
list: Expression,
start: Expression,
end: Expression,
}
impl AbstractTree for Sublist {
fn from_syntax_node(source: &str, node: tree_sitter::Node) -> crate::Result<Self> {
let list_node = node.child(0).unwrap();
let list = Expression::from_syntax_node(source, list_node)?;
let start_node = node.child(2).unwrap();
let start = Expression::from_syntax_node(source, start_node)?;
let end_node = node.child(4).unwrap();
let end = Expression::from_syntax_node(source, end_node)?;
Ok(Sublist { list, start, end })
}
fn run(&self, source: &str, context: &mut crate::Map) -> crate::Result<crate::Value> {
let value = self.list.run(source, context)?;
let list = value.as_list()?.items();
let start = self.start.run(source, context)?.as_integer()? as usize;
let end = self.end.run(source, context)?.as_integer()? as usize;
let sublist = &list[start..=end];
Ok(Value::List(List::with_items(sublist.to_vec())))
}
}

12
src/abstract_tree/transform.rs
View File

@ -36,18 +36,16 @@ impl AbstractTree for Transform {
let new_values = values
.par_iter()
.map(|value| {
let mut iter_context = Map::new();
let iter_context = Map::clone_from(context).unwrap();
iter_context
.variables_mut()
.unwrap()
.insert(key.clone(), value.clone());
let item_run = self.item.run(source, &mut iter_context);
match item_run {
Ok(value) => value,
Err(_) => Value::Empty,
}
self.item
.run(source, &mut iter_context.clone())
.unwrap_or_default()
})
.filter(|value| !value.is_empty())
.collect();

117
src/abstract_tree/type.rs Normal file
View File

@ -0,0 +1,117 @@
use std::fmt::{self, Display, Formatter};
use serde::{Deserialize, Serialize};
use tree_sitter::Node;
use crate::{AbstractTree, Error, Map, Result, Value};
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub enum Type {
Any,
Boolean,
Float,
Function,
Integer,
List,
Map,
String,
Table,
}
impl Type {
pub fn check(&self, value: &Value) -> Result<()> {
match (self, value.r#type()) {
(Type::Any, _)
| (Type::Boolean, Type::Boolean)
| (Type::Float, Type::Float)
| (Type::Function, Type::Function)
| (Type::Integer, Type::Integer)
| (Type::List, Type::List)
| (Type::Map, Type::Map)
| (Type::String, Type::String)
| (Type::Table, Type::Table) => Ok(()),
(Type::Boolean, _) => Err(Error::TypeCheck {
expected: Type::Boolean,
actual: value.clone(),
}),
(Type::Float, _) => Err(Error::TypeCheck {
expected: Type::Float,
actual: value.clone(),
}),
(Type::Function, _) => Err(Error::TypeCheck {
expected: Type::Function,
actual: value.clone(),
}),
(Type::Integer, _) => Err(Error::TypeCheck {
expected: Type::Integer,
actual: value.clone(),
}),
(Type::List, _) => Err(Error::TypeCheck {
expected: Type::List,
actual: value.clone(),
}),
(Type::Map, _) => Err(Error::TypeCheck {
expected: Type::Map,
actual: value.clone(),
}),
(Type::String, _) => Err(Error::TypeCheck {
expected: Type::String,
actual: value.clone(),
}),
(Type::Table, _) => Err(Error::TypeCheck {
expected: Type::Table,
actual: value.clone(),
}),
}
}
}
impl AbstractTree for Type {
fn from_syntax_node(source: &str, node: Node) -> Result<Self> {
Error::expect_syntax_node(source, "type", node)?;
let range_without_punctuation = node.start_byte() + 1..node.end_byte() - 1;
let r#type = match &source[range_without_punctuation] {
"any" => Type::Any,
"bool" => Type::Boolean,
"float" => Type::Float,
"fn" => Type::Function,
"int" => Type::Integer,
"list" => Type::List,
"map" => Type::Map,
"str" => Type::String,
"table" => Type::Table,
_ => {
return Err(Error::UnexpectedSyntaxNode {
expected: "any, bool, float, fn, int, list, map, str or table",
actual: node.kind(),
location: node.start_position(),
relevant_source: source[node.byte_range()].to_string(),
})
}
};
Ok(r#type)
}
fn run(&self, _source: &str, _context: &mut Map) -> Result<Value> {
Ok(Value::Empty)
}
}
impl Display for Type {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
match self {
Type::Any => write!(f, "any"),
Type::Boolean => write!(f, "bool"),
Type::Float => write!(f, "float"),
Type::Function => write!(f, "function"),
Type::Integer => write!(f, "integer"),
Type::List => write!(f, "list"),
Type::Map => write!(f, "map"),
Type::String => write!(f, "string"),
Type::Table => write!(f, "table"),
}
}
}

32
src/abstract_tree/use.rs
View File

@ -1,34 +1,32 @@
use std::fs::read_to_string;
use serde::{Deserialize, Serialize};
use tree_sitter::Node;
use crate::{evaluate_with_context, AbstractTree, Result, Value, ValueNode, VariableMap};
use crate::{evaluate_with_context, AbstractTree, Error, Map, Result, Value};
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub struct Use {
path: ValueNode,
path: String,
}
impl AbstractTree for Use {
fn from_syntax_node(source: &str, node: Node) -> Result<Self> {
let path_node = node.child(1).unwrap();
let value_node = ValueNode::from_syntax_node(source, path_node)?;
fn from_syntax_node(source: &str, node: tree_sitter::Node) -> crate::Result<Self> {
Error::expect_syntax_node(source, "use", node)?;
Ok(Use { path: value_node })
let string_node = node.child(1).unwrap();
let path = source[string_node.start_byte() + 1..string_node.end_byte() - 1].to_string();
println!("{path}");
Ok(Use { path })
}
fn run(&self, source: &str, context: &mut VariableMap) -> Result<Value> {
let run_node = self.path.run(source, context)?;
let path = run_node.as_string()?;
let file_contents = read_to_string(path)?;
let mut temp_context = VariableMap::new();
let eval_result = evaluate_with_context(&file_contents, &mut temp_context)?;
fn run(&self, _source: &str, _context: &mut Map) -> Result<Value> {
let file_contents = read_to_string(&self.path)?;
let mut file_context = Map::new();
while let Some((key, value)) = temp_context.inner_mut().pop_first() {
context.set_value(key, value)?;
}
evaluate_with_context(&file_contents, &mut file_context)?;
Ok(eval_result)
Ok(Value::Map(file_context))
}
}

143
src/abstract_tree/value_node.rs
View File

@ -1,32 +1,27 @@
use std::{collections::BTreeMap, ops::Range};
use std::collections::BTreeMap;
use serde::{Deserialize, Serialize};
use tree_sitter::Node;
use crate::{
AbstractTree, Block, Error, Expression, Function, Identifier, List, Map, Result, Statement,
Table, Value, ValueType,
AbstractTree, Error, Expression, Function, Identifier, List, Map, Result, Statement, Table,
Value,
};
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub struct ValueNode {
value_type: ValueType,
start_byte: usize,
end_byte: usize,
}
impl ValueNode {
pub fn new(value_type: ValueType, start_byte: usize, end_byte: usize) -> Self {
Self {
value_type,
start_byte,
end_byte,
}
}
pub fn byte_range(&self) -> Range<usize> {
self.start_byte..self.end_byte
}
pub enum ValueNode {
Boolean(String),
Float(String),
Integer(String),
String(String),
List(Vec<Expression>),
Empty,
Map(BTreeMap<String, Statement>),
Table {
column_names: Vec<Identifier>,
rows: Box<Expression>,
},
Function(Function),
}
impl AbstractTree for ValueNode {
@ -34,12 +29,15 @@ impl AbstractTree for ValueNode {
debug_assert_eq!("value", node.kind());
let child = node.child(0).unwrap();
let value_type = match child.kind() {
"integer" => ValueType::Integer,
"float" => ValueType::Float,
"string" => ValueType::String,
"boolean" => ValueType::Boolean,
"empty" => ValueType::Empty,
let value_node = match child.kind() {
"boolean" => ValueNode::Boolean(source[child.byte_range()].to_string()),
"float" => ValueNode::Float(source[child.byte_range()].to_string()),
"integer" => ValueNode::Integer(source[child.byte_range()].to_string()),
"string" => {
let without_quotes = child.start_byte() + 1..child.end_byte() - 1;
ValueNode::String(source[without_quotes].to_string())
}
"list" => {
let mut expressions = Vec::new();
@ -52,7 +50,7 @@ impl AbstractTree for ValueNode {
}
}
ValueType::List(expressions)
ValueNode::List(expressions)
}
"table" => {
let identifier_list_node = child.child(1).unwrap();
@ -72,7 +70,7 @@ impl AbstractTree for ValueNode {
let expression_node = child.child(2).unwrap();
let expression = Expression::from_syntax_node(source, expression_node)?;
ValueType::Table {
ValueNode::Table {
column_names,
rows: Box::new(expression),
}
@ -97,37 +95,17 @@ impl AbstractTree for ValueNode {
}
}
ValueType::Map(child_nodes)
ValueNode::Map(child_nodes)
}
"function" => {
let parameters_node = child.child_by_field_name("parameters");
let parameters = if let Some(node) = parameters_node {
let mut parameter_list = Vec::new();
for index in 0..node.child_count() {
let child_node = node.child(index).unwrap();
if child_node.is_named() {
let parameter = Identifier::from_syntax_node(source, child_node)?;
parameter_list.push(parameter);
}
}
Some(parameter_list)
} else {
None
};
let body_node = child.child_by_field_name("body").unwrap();
let body = Block::from_syntax_node(source, body_node)?;
ValueType::Function(Function::new(parameters, body))
}
"future" => {
let block_node = child.child(1).unwrap();
let block = Block::from_syntax_node(source, block_node)?;
ValueType::Future(block)
"function" => ValueNode::Function(Function::from_syntax_node(source, child)?),
_ => {
return Err(Error::UnexpectedSyntaxNode {
expected:
"string, integer, float, boolean, list, table, map, function or empty",
actual: child.kind(),
location: child.start_position(),
relevant_source: source[child.byte_range()].to_string(),
})
}
_ => return Err(Error::UnexpectedSyntaxNode {
expected:
@ -138,29 +116,19 @@ impl AbstractTree for ValueNode {
}),
};
Ok(ValueNode {
value_type,
start_byte: child.start_byte(),
end_byte: child.end_byte(),
})
Ok(value_node)
}
fn run(&self, source: &str, context: &mut Map) -> Result<Value> {
let value_source = &source[self.byte_range()];
let value = match &self.value_type {
ValueType::Any => todo!(),
ValueType::String => {
let without_quotes = &value_source[1..value_source.len() - 1];
let value = match self {
ValueNode::Boolean(value_source) => Value::Boolean(value_source.parse().unwrap()),
ValueNode::Float(value_source) => Value::Float(value_source.parse().unwrap()),
ValueNode::Integer(value_source) => Value::Integer(value_source.parse().unwrap()),
ValueNode::String(value_source) => Value::String(value_source.parse().unwrap()),
ValueNode::List(expressions) => {
let mut values = Vec::with_capacity(expressions.len());
Value::String(without_quotes.to_string())
}
ValueType::Float => Value::Float(value_source.parse().unwrap()),
ValueType::Integer => Value::Integer(value_source.parse().unwrap()),
ValueType::Boolean => Value::Boolean(value_source.parse().unwrap()),
ValueType::List(nodes) => {
let mut values = Vec::with_capacity(nodes.len());
for node in nodes {
for node in expressions {
let value = node.run(source, context)?;
values.push(value);
@ -168,19 +136,23 @@ impl AbstractTree for ValueNode {
Value::List(List::with_items(values))
}
ValueType::Empty => Value::Empty,
ValueType::Map(nodes) => {
ValueNode::Empty => Value::Empty,
ValueNode::Map(key_statement_pairs) => {
let map = Map::new();
for (key, node) in nodes {
let value = node.run(source, context)?;
{
let mut variables = map.variables_mut()?;
map.variables_mut().insert(key.clone(), value);
for (key, statement) in key_statement_pairs {
let value = statement.run(source, context)?;
variables.insert(key.clone(), value);
}
}
Value::Map(map)
}
ValueType::Table {
ValueNode::Table {
column_names,
rows: row_expression,
} => {
@ -206,8 +178,7 @@ impl AbstractTree for ValueNode {
Value::Table(table)
}
ValueType::Function(function) => Value::Function(function.clone()),
ValueType::Future(block) => Value::Future(block.clone()),
ValueNode::Function(function) => Value::Function(function.clone()),
};
Ok(value)

47
src/abstract_tree/yield.rs Normal file
View File

@ -0,0 +1,47 @@
use serde::{Deserialize, Serialize};
use tree_sitter::Node;
use crate::{AbstractTree, BuiltInFunction, Expression, FunctionCall, Result, Value};
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub struct Yield {
call: FunctionCall,
}
impl AbstractTree for Yield {
fn from_syntax_node(source: &str, node: Node) -> Result<Self> {
let input_node = node.child(0).unwrap();
let input = Expression::from_syntax_node(source, input_node)?;
let function_node = node.child(3).unwrap();
let mut arguments = Vec::new();
arguments.push(input);
for index in 4..node.child_count() - 1 {
let node = node.child(index).unwrap();
if node.is_named() {
let expression = Expression::from_syntax_node(source, node)?;
arguments.push(expression);
}
}
let call = if function_node.kind() == "built_in_function" {
let function = BuiltInFunction::from_syntax_node(source, function_node)?;
FunctionCall::BuiltIn(Box::new(function))
} else {
let name = Expression::from_syntax_node(source, function_node)?;
FunctionCall::ContextDefined { name, arguments }
};
Ok(Yield { call })
}
fn run(&self, source: &str, context: &mut crate::Map) -> Result<Value> {
self.call.run(source, context)
}
}

107
src/error.rs
View File

@ -3,9 +3,11 @@
//! To deal with errors from dependencies, either create a new error variant
//! or use the ToolFailure variant if the error can only occur inside a tool.
use crate::{value::Value, Identifier};
use tree_sitter::{Node, Point};
use std::{fmt, io, time, string::FromUtf8Error, num::ParseFloatError};
use crate::{value::Value, Identifier, Type};
use std::{fmt, io, num::ParseFloatError, string::FromUtf8Error, sync::PoisonError, time};
pub type Result<T> = std::result::Result<T, Error>;
@ -14,10 +16,15 @@ pub enum Error {
UnexpectedSyntaxNode {
expected: &'static str,
actual: &'static str,
location: tree_sitter::Point,
location: Point,
relevant_source: String,
},
TypeCheck {
expected: Type,
actual: Value,
},
/// The 'assert' macro did not resolve successfully.
AssertEqualFailed {
expected: Value,
@ -57,7 +64,7 @@ pub enum Error {
actual: Value,
},
ExpectedInt {
ExpectedInteger {
actual: Value,
},
@ -125,9 +132,33 @@ pub enum Error {
/// A custom error explained by its message.
CustomMessage(String),
/// Invalid user input.
Syntax {
source: String,
location: Point,
},
}
impl Error {
pub fn expect_syntax_node(source: &str, expected: &'static str, actual: Node) -> Result<()> {
if expected == actual.kind() {
Ok(())
} else if actual.is_error() {
Err(Error::Syntax {
source: source[actual.byte_range()].to_string(),
location: actual.start_position(),
})
} else {
Err(Error::UnexpectedSyntaxNode {
expected,
actual: actual.kind(),
location: actual.start_position(),
relevant_source: source[actual.byte_range()].to_string(),
})
}
}
pub fn expect_tool_argument_amount(
tool_name: &'static str,
expected: usize,
@ -145,6 +176,12 @@ impl Error {
}
}
impl<T> From<PoisonError<T>> for Error {
fn from(value: PoisonError<T>) -> Self {
Error::ToolFailure(value.to_string())
}
}
impl From<FromUtf8Error> for Error {
fn from(value: FromUtf8Error) -> Self {
Error::ToolFailure(value.to_string())
@ -163,12 +200,6 @@ impl From<csv::Error> for Error {
}
}
impl From<json::Error> for Error {
fn from(value: json::Error) -> Self {
Error::ToolFailure(value.to_string())
}
}
impl From<io::Error> for Error {
fn from(value: std::io::Error) -> Self {
Error::ToolFailure(value.to_string())
@ -217,7 +248,7 @@ impl fmt::Display for Error {
if expected.is_table() {
write!(f, "\n{expected}\n")?;
} else {
} else {
write!(f, " {expected} ")?;
}
@ -225,10 +256,10 @@ impl fmt::Display for Error {
if actual.is_table() {
write!(f, "\n{actual}")
} else {
} else {
write!(f, " {actual}.")
}
},
}
AssertFailed => write!(
f,
"Assertion failed. A false value was passed to \"assert\"."
@ -255,24 +286,20 @@ impl fmt::Display for Error {
"{identifier} expected a minimum of {minimum} arguments, but got {actual}.",
),
ExpectedString { actual } => {
write!(f, "Expected a Value::String, but got {:?}.", actual)
write!(f, "Expected a string but got {:?}.", actual)
}
ExpectedInteger { actual } => write!(f, "Expected an integer, but got {:?}.", actual),
ExpectedFloat { actual } => write!(f, "Expected a float, but got {:?}.", actual),
ExpectedNumber { actual } => {
write!(f, "Expected a float or integer but got {:?}.", actual)
}
ExpectedNumberOrString { actual } => {
write!(f, "Expected a number or string, but got {:?}.", actual)
}
ExpectedInt { actual } => write!(f, "Expected a Value::Int, but got {:?}.", actual),
ExpectedFloat { actual } => write!(f, "Expected a Value::Float, but got {:?}.", actual),
ExpectedNumber { actual } => write!(
f,
"Expected a Value::Float or Value::Int, but got {:?}.",
actual
),
ExpectedNumberOrString { actual } => write!(
f,
"Expected a Value::Number or a Value::String, but got {:?}.",
actual
),
ExpectedBoolean { actual } => {
write!(f, "Expected a Value::Boolean, but got {:?}.", actual)
write!(f, "Expected a boolean, but got {:?}.", actual)
}
ExpectedList { actual } => write!(f, "Expected a Value::List, but got {:?}.", actual),
ExpectedList { actual } => write!(f, "Expected a list, but got {:?}.", actual),
ExpectedMinLengthList {
minimum_len,
actual_len,
@ -285,14 +312,14 @@ impl fmt::Display for Error {
actual,
} => write!(
f,
"Expected a Value::List of len {}, but got {:?}.",
"Expected a list of len {}, but got {:?}.",
expected_len, actual
),
ExpectedEmpty { actual } => write!(f, "Expected a Value::Empty, but got {:?}.", actual),
ExpectedMap { actual } => write!(f, "Expected a Value::Map, but got {:?}.", actual),
ExpectedTable { actual } => write!(f, "Expected a Value::Table, but got {:?}.", actual),
ExpectedEmpty { actual } => write!(f, "Expected an empty value, but got {:?}.", actual),
ExpectedMap { actual } => write!(f, "Expected a map, but got {:?}.", actual),
ExpectedTable { actual } => write!(f, "Expected a table, but got {:?}.", actual),
ExpectedFunction { actual } => {
write!(f, "Expected Value::Function, but got {:?}.", actual)
write!(f, "Expected function, but got {:?}.", actual)
}
ExpectedCollection { actual } => {
write!(
@ -318,11 +345,21 @@ impl fmt::Display for Error {
relevant_source,
} => write!(
f,
"Expected syntax node {expected}, but got {actual} at {location}. Code: {relevant_source} ",
"Expected {expected}, but got {actual} at {location}. Code: {relevant_source} ",
),
WrongColumnAmount { expected, actual } => write!(
f,
"Wrong column amount. Expected {expected} but got {actual}."
),
WrongColumnAmount { expected, actual } => write!(f, "Wrong column amount. Expected {expected} but got {actual}."),
ToolFailure(message) => write!(f, "{message}"),
CustomMessage(message) => write!(f, "{message}"),
Syntax { source, location } => {
write!(f, "Syntax error at {location}, this is not valid: {source}")
}
TypeCheck { expected, actual } => write!(
f,
"Type check error. Expected a {expected} but got {actual}."
),
}
}
}

58
src/evaluator.rs
View File

@ -6,7 +6,7 @@ use std::fmt::{self, Debug, Formatter};
use tree_sitter::{Parser, Tree as TSTree};
use crate::{language, AbstractTree, Block, Map, Result, Value};
use crate::{language, AbstractTree, Map, Result, Statement, Value};
/// Evaluate the given source code.
///
@ -33,9 +33,13 @@ pub fn evaluate(source: &str) -> Result<Value> {
/// # use dust_lang::*;
/// let mut context = Map::new();
///
/// context.set_value("one".into(), 1.into());
/// context.set_value("two".into(), 2.into());
/// context.set_value("three".into(), 3.into());
/// {
/// let mut variables = context.variables_mut().unwrap();
///
/// variables.insert("one".into(), 1.into());
/// variables.insert("two".into(), 2.into());
/// variables.insert("three".into(), 3.into());
/// }
///
/// let dust_code = "four = 4 one + two + three + four";
///
@ -55,10 +59,10 @@ pub fn evaluate_with_context(source: &str, context: &mut Map) -> Result<Value> {
///
/// The Evaluator turns a tree sitter concrete syntax tree into a vector of
/// abstract trees called [Item][]s that can be run to execute the source code.
pub struct Evaluator<'context, 'code> {
pub struct Evaluator<'c, 's> {
_parser: Parser,
context: &'context mut Map,
source: &'code str,
context: &'c mut Map,
source: &'s str,
syntax_tree: TSTree,
}
@ -68,8 +72,8 @@ impl Debug for Evaluator<'_, '_> {
}
}
impl<'context, 'code> Evaluator<'context, 'code> {
fn new(mut parser: Parser, context: &'context mut Map, source: &'code str) -> Self {
impl<'c, 's> Evaluator<'c, 's> {
pub fn new(mut parser: Parser, context: &'c mut Map, source: &'s str) -> Self {
let syntax_tree = parser.parse(source, None).unwrap();
Evaluator {
@ -80,17 +84,23 @@ impl<'context, 'code> Evaluator<'context, 'code> {
}
}
fn run(self) -> Result<Value> {
let mut cursor = self.syntax_tree.walk();
let root_node = cursor.node();
let mut prev_result = Ok(Value::Empty);
pub fn run(self) -> Result<Value> {
let root_node = self.syntax_tree.root_node();
for block_node in root_node.children(&mut cursor) {
let block = Block::from_syntax_node(self.source, block_node)?;
prev_result = block.run(self.source, self.context);
let mut prev_result = Value::Empty;
for index in 0..root_node.child_count() {
let statement_node = root_node.child(index).unwrap();
let statement = Statement::from_syntax_node(self.source, statement_node)?;
prev_result = statement.run(self.source, self.context)?;
}
prev_result
Ok(prev_result)
}
pub fn syntax_tree(&self) -> String {
self.syntax_tree.root_node().to_sexp()
}
}
@ -149,10 +159,12 @@ mod tests {
fn evaluate_map() {
let map = Map::new();
map.variables_mut()
.insert("x".to_string(), Value::Integer(1));
map.variables_mut()
.insert("foo".to_string(), Value::String("bar".to_string()));
{
let mut variables = map.variables_mut().unwrap();
variables.insert("x".to_string(), Value::Integer(1));
variables.insert("foo".to_string(), Value::String("bar".to_string()));
}
assert_eq!(evaluate("{ x = 1, foo = 'bar' }"), Ok(Value::Map(map)));
}
@ -221,7 +233,7 @@ mod tests {
}
#[test]
fn evaluate_if_else_else_if_else_if_else_if_else() {
fn evaluate_if_else_if_else_if_else_if_else() {
assert_eq!(
evaluate(
"
@ -247,7 +259,7 @@ mod tests {
assert_eq!(
evaluate(
"
foobar = |message| => message
foobar = |message <str>| <str> { message }
(foobar 'Hiya')
",
),

2
src/lib.rs
View File

@ -8,7 +8,7 @@ pub use crate::{
abstract_tree::*,
error::*,
evaluator::*,
value::{function::Function, list::List, map::Map, table::Table, value_type::ValueType, Value},
value::{function::Function, list::List, map::Map, table::Table, Value},
};
mod abstract_tree;

30
src/main.rs
View File

@ -1,5 +1,4 @@
//! Command line interface for the dust programming language.
use async_std::fs::read_to_string;
use clap::Parser;
use rustyline::{
completion::FilenameCompleter,
@ -9,10 +8,11 @@ use rustyline::{
history::DefaultHistory,
Completer, Context, Editor, Helper, Validator,
};
use tree_sitter::Parser as TSParser;
use std::borrow::Cow;
use std::{borrow::Cow, fs::read_to_string};
use dust_lang::{evaluate_with_context, Map, Value};
use dust_lang::{evaluate_with_context, language, Evaluator, Map, Value};
/// Command-line arguments to be parsed.
#[derive(Parser, Debug)]
@ -30,12 +30,15 @@ struct Args {
#[arg(short = 'p', long)]
input_path: Option<String>,
/// A path to file whose contents will be assigned to the "input" variable.
#[arg(short = 't', long = "tree")]
show_syntax_tree: bool,
/// Location of the file to run.
path: Option<String>,
}
#[async_std::main]
async fn main() {
fn main() {
let args = Args::parse();
if args.path.is_none() && args.command.is_none() {
@ -43,7 +46,7 @@ async fn main() {
}
let source = if let Some(path) = &args.path {
read_to_string(path).await.unwrap()
read_to_string(path).unwrap()
} else if let Some(command) = &args.command {
command.clone()
} else {
@ -55,18 +58,29 @@ async fn main() {
if let Some(input) = args.input {
context
.variables_mut()
.unwrap()
.insert("input".to_string(), Value::String(input));
}
if let Some(path) = args.input_path {
let file_contents = read_to_string(path).await.unwrap();
let file_contents = read_to_string(path).unwrap();
context
.variables_mut()
.unwrap()
.insert("input".to_string(), Value::String(file_contents));
}
let eval_result = evaluate_with_context(&source, &mut context);
let mut parser = TSParser::new();
parser.set_language(language()).unwrap();
let evaluator = Evaluator::new(parser, &mut context, &source);
if args.show_syntax_tree {
println!("{}", evaluator.syntax_tree());
}
let eval_result = evaluator.run();
match eval_result {
Ok(value) => {

82
src/value/function.rs
View File

@ -1,29 +1,81 @@
use std::fmt::{self, Display, Formatter};
use serde::{Deserialize, Serialize};
use tree_sitter::Node;
use crate::{Block, Identifier};
use crate::{AbstractTree, Block, Error, Identifier, Map, Result, Type, Value};
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, PartialOrd, Ord)]
pub struct Function {
parameters: Option<Vec<Identifier>>,
body: Box<Block>,
parameters: Vec<(Identifier, Type)>,
return_type: Option<Type>,
body: Block,
}
impl Function {
pub fn new(parameters: Option<Vec<Identifier>>, body: Block) -> Self {
Function {
parameters,
body: Box::new(body),
}
}
pub fn identifiers(&self) -> &Option<Vec<Identifier>> {
pub fn parameters(&self) -> &Vec<(Identifier, Type)> {
&self.parameters
}
}
pub fn body(&self) -> &Block {
&self.body
impl AbstractTree for Function {
fn from_syntax_node(source: &str, node: Node) -> Result<Self> {
Error::expect_syntax_node(source, "function", node)?;
let child_count = node.child_count();
let mut parameters = Vec::new();
for index in 1..child_count - 2 {
let parameter_node = {
let child = node.child(index).unwrap();
if child.is_named() {
child
} else {
continue;
}
};
Error::expect_syntax_node(source, "parameter", parameter_node)?;
let identifier_node = parameter_node.child(0).unwrap();
let identifier = Identifier::from_syntax_node(source, identifier_node)?;
let type_node = parameter_node.child(1).unwrap();
let r#type = Type::from_syntax_node(source, type_node)?;
parameters.push((identifier, r#type))
}
let return_type_node = node.child(child_count - 2).unwrap();
let return_type = if return_type_node.is_named() {
Some(Type::from_syntax_node(source, return_type_node)?)
} else {
None
};
let body_node = node.child(child_count - 1).unwrap();
let body = Block::from_syntax_node(source, body_node)?;
Ok(Function {
parameters,
return_type,
body,
})
}
fn run(&self, source: &str, context: &mut Map) -> Result<Value> {
let return_value = self.body.run(source, context)?;
if let Some(r#type) = &self.return_type {
r#type.check(&return_value)?;
} else if !return_value.is_empty() {
return Err(Error::ExpectedEmpty {
actual: return_value.clone(),
});
}
Ok(return_value)
}
}
@ -31,8 +83,8 @@ impl Display for Function {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
write!(
f,
"function < {:?} > {{ {:?} }}", // TODO: Correct this output
self.parameters, self.body
"Function {{ parameters: {:?}, return_type: {:?}, body: {:?} }}",
self.parameters, self.return_type, self.body
)
}
}

36
src/value/map.rs
View File

@ -6,7 +6,7 @@ use std::{
sync::{Arc, RwLock, RwLockReadGuard, RwLockWriteGuard},
};
use crate::{value::Value, List, Table};
use crate::{value::Value, List, Result, Table};
/// A collection dust variables comprised of key-value pairs.
///
@ -25,34 +25,24 @@ impl Map {
}
}
pub fn clone_from(other: &Self) -> Self {
pub fn clone_from(other: &Self) -> Result<Self> {
let mut new_map = BTreeMap::new();
for (key, value) in other.variables().iter() {
for (key, value) in other.variables()?.iter() {
new_map.insert(key.clone(), value.clone());
}
Map {
Ok(Map {
variables: Arc::new(RwLock::new(new_map)),
}
})
}
pub fn variables(&self) -> RwLockReadGuard<BTreeMap<String, Value>> {
self.variables.read().unwrap()
pub fn variables(&self) -> Result<RwLockReadGuard<BTreeMap<String, Value>>> {
Ok(self.variables.read()?)
}
pub fn variables_mut(&self) -> RwLockWriteGuard<BTreeMap<String, Value>> {
self.variables.write().unwrap()
}
/// Returns the number of stored variables.
pub fn len(&self) -> usize {
self.variables.read().unwrap().len()
}
/// Returns true if the length is zero.
pub fn is_empty(&self) -> bool {
self.variables.read().unwrap().is_empty()
pub fn variables_mut(&self) -> Result<RwLockWriteGuard<BTreeMap<String, Value>>> {
Ok(self.variables.write()?)
}
}
@ -104,12 +94,12 @@ impl Display for Map {
}
}
impl From<&Table> for Map {
fn from(value: &Table) -> Self {
impl From<&Table> for Result<Map> {
fn from(value: &Table) -> Result<Map> {
let map = Map::new();
for (row_index, row) in value.rows().iter().enumerate() {
map.variables_mut()
map.variables_mut()?
.insert(
row_index.to_string(),
Value::List(List::with_items(row.clone())),
@ -117,7 +107,7 @@ impl From<&Table> for Map {
.unwrap();
}
map
Ok(map)
}
}

142
src/value/mod.rs
View File

@ -1,10 +1,9 @@
//! Types that represent runtime values.
use crate::{
error::{Error, Result},
Block, Function, List, Map, Table, ValueType,
Function, List, Map, Table, Type,
};
use json::JsonValue;
use serde::{
de::{MapAccess, SeqAccess, Visitor},
ser::SerializeTuple,
@ -23,11 +22,10 @@ pub mod function;
pub mod list;
pub mod map;
pub mod table;
pub mod value_type;
/// Whale value representation.
/// Dust value representation.
///
/// Every whale variable has a key and a Value. Variables are represented by
/// Every dust variable has a key and a Value. Variables are represented by
/// storing them in a VariableMap. This means the map of variables is itself a
/// value that can be treated as any other.
#[derive(Debug, Clone, Default)]
@ -46,8 +44,18 @@ pub enum Value {
}
impl Value {
pub fn value_type(&self) -> ValueType {
ValueType::from(self)
pub fn r#type(&self) -> Type {
match self {
Value::List(_) => Type::List,
Value::Map(_) => Type::Map,
Value::Table(_) => Type::Table,
Value::Function(_) => Type::Function,
Value::String(_) => Type::String,
Value::Float(_) => Type::Float,
Value::Integer(_) => Type::Integer,
Value::Boolean(_) => Type::Boolean,
Value::Empty => Type::Any,
}
}
pub fn is_table(&self) -> bool {
@ -104,7 +112,7 @@ impl Value {
pub fn as_integer(&self) -> Result<i64> {
match self {
Value::Integer(i) => Ok(*i),
value => Err(Error::ExpectedInt {
value => Err(Error::ExpectedInteger {
actual: value.clone(),
}),
}
@ -208,7 +216,7 @@ impl Value {
match self {
Value::Table(table) => Ok(table.clone()),
Value::List(list) => Ok(Table::from(list)),
Value::Map(map) => Ok(Table::from(map)),
Value::Map(map) => Result::from(map),
value => Err(Error::ExpectedTable {
actual: value.clone(),
}),
@ -220,19 +228,20 @@ impl Add for Value {
type Output = Result<Value>;
fn add(self, other: Self) -> Self::Output {
match (self.as_integer(), other.as_integer()) {
(Ok(left), Ok(right)) => return Ok(Value::Integer(left + right)),
_ => {}
if let (Ok(left), Ok(right)) = (self.as_integer(), other.as_integer()) {
return Ok(Value::Integer(left + right));
}
match (self.as_number(), other.as_number()) {
(Ok(left), Ok(right)) => return Ok(Value::Float(left + right)),
_ => {}
if let (Ok(left), Ok(right)) = (self.as_number(), other.as_number()) {
return Ok(Value::Float(left + right));
}
match (self.as_string(), other.as_string()) {
(Ok(left), Ok(right)) => return Ok(Value::String(left.to_string() + right)),
_ => {}
if let (Ok(left), Ok(right)) = (self.as_string(), other.as_string()) {
return Ok(Value::String(left.to_string() + right));
}
if self.is_string() || other.is_string() {
return Ok(Value::String(self.to_string() + &other.to_string()));
}
let non_number_or_string = if !self.is_number() == !self.is_string() {
@ -335,6 +344,17 @@ impl SubAssign for Value {
(Value::Integer(left), Value::Integer(right)) => *left -= right,
(Value::Float(left), Value::Float(right)) => *left -= right,
(Value::Float(left), Value::Integer(right)) => *left -= right as f64,
(Value::List(list), value) => {
let index_to_remove = list
.items()
.iter()
.enumerate()
.find_map(|(i, list_value)| if list_value == &value { Some(i) } else { None });
if let Some(index) = index_to_remove {
list.items_mut().remove(index);
}
}
_ => {}
}
}
@ -500,82 +520,6 @@ impl From<()> for Value {
}
}
impl TryFrom<JsonValue> for Value {
type Error = Error;
fn try_from(json_value: JsonValue) -> Result<Self> {
use JsonValue::*;
match json_value {
Null => Ok(Value::Empty),
Short(short) => Ok(Value::String(short.to_string())),
String(string) => Ok(Value::String(string)),
Number(number) => Ok(Value::Float(f64::from(number))),
Boolean(boolean) => Ok(Value::Boolean(boolean)),
Object(object) => {
let map = Map::new();
for (key, node_value) in object.iter() {
let value = Value::try_from(node_value)?;
map.variables_mut().insert(key.to_string(), value);
}
Ok(Value::Map(map))
}
Array(array) => {
let mut values = Vec::new();
for json_value in array {
let value = Value::try_from(json_value)?;
values.push(value);
}
Ok(Value::List(List::with_items(values)))
}
}
}
}
impl TryFrom<&JsonValue> for Value {
type Error = Error;
fn try_from(json_value: &JsonValue) -> Result<Self> {
use JsonValue::*;
match json_value {
Null => Ok(Value::Empty),
Short(short) => Ok(Value::String(short.to_string())),
String(string) => Ok(Value::String(string.clone())),
Number(number) => Ok(Value::Float(f64::from(*number))),
Boolean(boolean) => Ok(Value::Boolean(*boolean)),
Object(object) => {
let map = Map::new();
for (key, node_value) in object.iter() {
let value = Value::try_from(node_value)?;
map.variables_mut().insert(key.to_string(), value);
}
Ok(Value::Map(map))
}
Array(array) => {
let mut values = Vec::new();
for json_value in array {
let value = Value::try_from(json_value)?;
values.push(value);
}
Ok(Value::List(List::with_items(values)))
}
}
}
}
impl TryFrom<Value> for String {
type Error = Error;
@ -607,7 +551,7 @@ impl TryFrom<Value> for i64 {
if let Value::Integer(value) = value {
Ok(value)
} else {
Err(Error::ExpectedInt { actual: value })
Err(Error::ExpectedInteger { actual: value })
}
}
}
@ -846,8 +790,12 @@ impl<'de> Visitor<'de> for ValueVisitor {
{
let map = Map::new();
while let Some((key, value)) = access.next_entry()? {
map.variables_mut().insert(key, value);
{
let mut variables = map.variables_mut().unwrap();
while let Some((key, value)) = access.next_entry()? {
variables.insert(key, value);
}
}
Ok(Value::Map(map))

30
src/value/table.rs
View File

@ -158,7 +158,7 @@ impl Display for Table {
string
}
Value::Map(map) => format!("Map ({} items)", map.len()),
Value::Map(map) => format!("Map ({} items)", map.variables().unwrap().len()),
Value::Table(table) => format!("Table ({} items)", table.len()),
Value::Function(_) => "Function".to_string(),
Value::Empty => "Empty".to_string(),
@ -233,7 +233,7 @@ impl From<&Value> for Table {
}
Value::List(list) => Self::from(list),
Value::Empty => Table::new(Vec::with_capacity(0)),
Value::Map(map) => Self::from(map),
Value::Map(map) => Result::<Table>::from(map).unwrap(),
Value::Table(table) => table.clone(),
Value::Function(function) => {
let mut table = Table::new(vec!["function".to_string()]);
@ -287,39 +287,35 @@ impl From<&mut List> for Table {
}
}
impl From<Map> for Table {
impl From<Map> for Result<Table> {
fn from(map: Map) -> Self {
let variables = map.variables();
let variables = map.variables()?;
let keys = variables.keys().cloned().collect();
let values = variables.values().cloned().collect();
let mut table = Table::new(keys);
table
.insert(values)
.expect("Failed to create Table from Map. This is a no-op.");
table.insert(values)?;
table
Ok(table)
}
}
impl From<&Map> for Table {
impl From<&Map> for Result<Table> {
fn from(map: &Map) -> Self {
let variables = map.variables();
let variables = map.variables()?;
let keys = variables.keys().cloned().collect();
let values = variables.values().cloned().collect();
let mut table = Table::new(keys);
table
.insert(values)
.expect("Failed to create Table from Map. This is a no-op.");
table.insert(values)?;
table
Ok(table)
}
}
impl From<&mut Map> for Table {
impl From<&mut Map> for Result<Table> {
fn from(map: &mut Map) -> Self {
let variables = map.variables();
let variables = map.variables()?;
let keys = variables.keys().cloned().collect();
let values = variables.values().cloned().collect();
let mut table = Table::new(keys);
@ -328,7 +324,7 @@ impl From<&mut Map> for Table {
.insert(values)
.expect("Failed to create Table from Map. This is a no-op.");
table
Ok(table)
}
}

23
std/list_functions.ds Normal file
View File

@ -0,0 +1,23 @@
find = |items <list> function <fn>| <any> {
for i in items {
if (function i) {
return i
}
}
}
map = |items <list> function <fn>| <list> {
new_list = []
for i in items {
new_list += (function i)
}
new_list
}
foobar <int> = [0 1 2]
-> (map |i <int>| <int> { i - 1 })
-> (find |i <int>| <bool> { i == -1 })
foobar

75
tests/dust_examples.rs
View File

@ -2,6 +2,21 @@ use std::fs::read_to_string;
use dust_lang::*;
#[test]
fn r#async() {
let file_contents = read_to_string("examples/async.ds").unwrap();
evaluate(&file_contents).unwrap();
}
#[test]
#[ignore]
fn async_download() {
let file_contents = read_to_string("examples/async_download.ds").unwrap();
evaluate(&file_contents).unwrap();
}
#[test]
fn clue_solver() {
let file_contents = read_to_string("examples/clue_solver.ds").unwrap();
@ -24,13 +39,6 @@ fn fibonacci() {
evaluate(&file_contents).unwrap();
}
#[test]
fn find_loop() {
let file_contents = read_to_string("examples/find_loop.ds").unwrap();
evaluate(&file_contents).unwrap();
}
#[test]
fn fizzbuzz() {
let file_contents = read_to_string("examples/fizzbuzz.ds").unwrap();
@ -53,8 +61,43 @@ fn hello_world() {
}
#[test]
fn remove_loop() {
let file_contents = read_to_string("examples/remove_loop.ds").unwrap();
fn jq_data() {
let file_contents = read_to_string("examples/jq_data.ds").unwrap();
evaluate(&file_contents).unwrap();
}
#[test]
fn list() {
let file_contents = read_to_string("examples/list.ds").unwrap();
evaluate(&file_contents).unwrap();
}
#[test]
fn map() {
let file_contents = read_to_string("examples/map.ds").unwrap();
evaluate(&file_contents).unwrap();
}
#[test]
fn random() {
let file_contents = read_to_string("examples/random.ds").unwrap();
evaluate(&file_contents).unwrap();
}
#[test]
fn sea_creatures() {
let file_contents = read_to_string("examples/sea_creatures.ds").unwrap();
evaluate(&file_contents).unwrap();
}
#[test]
fn select() {
let file_contents = read_to_string("examples/select.ds").unwrap();
evaluate(&file_contents).unwrap();
}
@ -73,13 +116,6 @@ fn table() {
evaluate(&file_contents).unwrap();
}
#[test]
fn transform_loop() {
let file_contents = read_to_string("examples/transform_loop.ds").unwrap();
evaluate(&file_contents).unwrap();
}
#[test]
fn variables() {
let file_contents = read_to_string("examples/variables.ds").unwrap();
@ -93,3 +129,10 @@ fn while_loop() {
evaluate(&file_contents).unwrap();
}
#[test]
fn r#yield() {
let file_contents = read_to_string("examples/yield.ds").unwrap();
evaluate(&file_contents).unwrap();
}

79
tree-sitter-dust/corpus/assignment.txt Normal file
View File

@ -0,0 +1,79 @@
================================================================================
Simple Assignment
================================================================================
x = y
--------------------------------------------------------------------------------
(root
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(identifier))))))
================================================================================
Simple Assignment with Type
================================================================================
x <int> = y
--------------------------------------------------------------------------------
(root
(statement
(assignment
(identifier)
(type)
(assignment_operator)
(statement
(expression
(identifier))))))
================================================================================
Map Item Assignment
================================================================================
x:y = 1
--------------------------------------------------------------------------------
(root
(statement
(index_assignment
(index
(expression
(identifier))
(expression
(identifier)))
(assignment_operator)
(statement
(expression
(value
(integer)))))))
================================================================================
List Item Assignment
================================================================================
x:9 = 'foobar'
--------------------------------------------------------------------------------
(root
(statement
(index_assignment
(index
(expression
(identifier))
(expression
(value
(integer))))
(assignment_operator)
(statement
(expression
(value
(string)))))))

71
tree-sitter-dust/corpus/async.txt Normal file
View File

@ -0,0 +1,71 @@
================================================================================
Simple Async Statements
================================================================================
async { (output 'Whaddup') }
--------------------------------------------------------------------------------
(root
(statement
(block
(statement
(expression
(function_call
(built_in_function
(expression
(value
(string))))))))))
================================================================================
Complex Async Statements
================================================================================
async {
if 1 % 2 == 0 {
true
} else {
false
}
'foobar'
}
--------------------------------------------------------------------------------
(root
(statement
(block
(statement
(if_else
(if
(expression
(logic
(expression
(math
(expression
(value
(integer)))
(math_operator)
(expression
(value
(integer)))))
(logic_operator)
(expression
(value
(integer)))))
(block
(statement
(expression
(value
(boolean))))))
(else
(block
(statement
(expression
(value
(boolean))))))))
(statement
(expression
(value
(string)))))))

59
tree-sitter-dust/corpus/built_in_function.txt
View File

@ -1,49 +1,36 @@
==================
================================================================================
Simple Function Call
==================
================================================================================
(output 'hi')
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(function_call
(built_in_function
(expression
(value
(string)))))))))
(statement
(expression
(function_call
(built_in_function
(expression
(value
(string))))))))
==================
================================================================================
Nested Function Call
==================
================================================================================
(assert_equal (random_integer) 4)
assert_equal random_integer 4
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(function_call
(built_in_function
(expression
(function_call
(built_in_function)))
(expression
(value
(integer)))))))
(statement
(expression
(function_call
(built_in_function
(expression
(function_call
(built_in_function
(expression
(value
(integer))))))))))))
(statement
(expression
(function_call
(built_in_function
(expression
(function_call
(built_in_function)))
(expression
(value
(integer))))))))

52
tree-sitter-dust/corpus/comments.txt
View File

@ -1,53 +1,45 @@
==================
================================================================================
Full Line Comments
==================
================================================================================
not_a_comment
# comment
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(identifier))))
(comment))
(statement
(expression
(identifier))))
==================
================================================================================
Partial Line Comments
==================
================================================================================
not_a_comment # comment
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(identifier))))
(comment))
(statement
(expression
(identifier))))
==================
================================================================================
Multiline Comments
==================
================================================================================
# comment #
not_a_comment #
# comment # "not a comment"
---
--------------------------------------------------------------------------------
(root
(comment)
(block
(statement
(expression
(identifier)))
(comment)
(comment)
(statement
(expression
(value
(string))))))
(statement
(expression
(identifier)))
(statement
(expression
(value
(string)))))

73
tree-sitter-dust/corpus/filter.txt
View File

@ -1,73 +0,0 @@
==================
Simple Filter Loop
==================
filter i in [1, 2, 3] {
i <= 1
}
---
(root
(block
(statement
(filter
(identifier)
(expression
(value
(list
(expression
(value
(integer)))
(expression
(value
(integer)))
(expression
(value
(integer))))))
(block
(statement
(expression
(logic
(expression
(identifier))
(logic_operator)
(expression
(value
(integer)))))))))))
==================
Nested Filter Loop
==================
filter i in big_list {
filter j in i {
i != 42
}
}
---
(root
(block
(statement
(filter
(identifier)
(expression
(identifier))
(block
(statement
(filter
(identifier)
(expression
(identifier))
(block
(statement
(expression
(logic
(expression
(identifier))
(logic_operator)
(expression
(value
(integer))))))))))))))

119
tree-sitter-dust/corpus/find.txt
View File

@ -1,119 +0,0 @@
================================================================================
Simple Find Loop
================================================================================
find i in [1, 2, 3] {
i <= 3
}
--------------------------------------------------------------------------------
(root
(block
(statement
(find
(identifier)
(expression
(value
(list
(expression
(value
(integer)))
(expression
(value
(integer)))
(expression
(value
(integer))))))
(block
(statement
(expression
(logic
(expression
(identifier))
(logic_operator)
(expression
(value
(integer)))))))))))
================================================================================
Nested Find Loop
================================================================================
find i in ["one", "two", "three"] {
found = find j in i {
i == "e"
}
if (type found) != 'empty' {
true
} else {
false
}
}
--------------------------------------------------------------------------------
(root
(block
(statement
(find
(identifier)
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(string)))
(expression
(value
(string))))))
(block
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(find
(identifier)
(expression
(identifier))
(block
(statement
(expression
(logic
(expression
(identifier))
(logic_operator)
(expression
(value
(string)))))))))))
(statement
(if_else
(if
(expression
(logic
(expression
(function_call
(built_in_function
(expression
(identifier)))))
(logic_operator)
(expression
(value
(string)))))
(block
(statement
(expression
(value
(boolean))))))
(else
(block
(statement
(expression
(value
(boolean)))))))))))))

90
tree-sitter-dust/corpus/for.txt
View File

@ -2,60 +2,64 @@
Simple For Loop
================================================================================
for i in [1, 2, 3] output i
for i in [1, 2, 3] {
(output i)
}
--------------------------------------------------------------------------------
(root
(block
(statement
(for
(identifier)
(expression
(value
(list
(expression
(value
(integer)))
(expression
(value
(integer)))
(expression
(value
(integer))))))
(block
(statement
(statement
(for
(identifier)
(expression
(value
(list
(expression
(function_call
(built_in_function
(expression
(identifier)))))))))))
(value
(integer)))
(expression
(value
(integer)))
(expression
(value
(integer))))))
(block
(statement
(expression
(function_call
(built_in_function
(expression
(identifier))))))))))
================================================================================
Nested For Loop
================================================================================
for list in list_of_lists for item in list output item
for list in list_of_lists {
for item in list {
(output item)
}
}
--------------------------------------------------------------------------------
(root
(block
(statement
(for
(identifier)
(expression
(identifier))
(block
(statement
(for
(identifier)
(expression
(identifier))
(block
(statement
(expression
(function_call
(built_in_function
(expression
(identifier))))))))))))))
(statement
(for
(identifier)
(expression
(identifier))
(block
(statement
(for
(identifier)
(expression
(identifier))
(block
(statement
(expression
(function_call
(built_in_function
(expression
(identifier)))))))))))))

169
tree-sitter-dust/corpus/functions.txt
View File

@ -2,45 +2,45 @@
Simple Function
================================================================================
=> "Hiya"
|| <str> { "Hiya" }
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(value
(function
(block
(statement
(expression
(value
(string)))))))))))
(statement
(expression
(value
(function
(type)
(block
(statement
(expression
(value
(string))))))))))
================================================================================
Function Assignment
================================================================================
x = => "Hiya"
x = || <str> { "Hiya" }
--------------------------------------------------------------------------------
(root
(block
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(function
(block
(statement
(expression
(value
(string)))))))))))))
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(function
(type)
(block
(statement
(expression
(value
(string))))))))))))
================================================================================
Function Call
@ -51,20 +51,20 @@ Function Call
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(function_call
(identifier)
(expression
(value
(string))))))))
(statement
(expression
(function_call
(expression
(identifier))
(expression
(value
(string)))))))
================================================================================
Complex Function
================================================================================
|message number| => {
|message <str> number <int>| {
(output message)
(output number)
}
@ -72,27 +72,29 @@ Complex Function
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(value
(function
(identifier_list
(identifier)
(identifier))
(block
(statement
(expression
(function_call
(built_in_function
(expression
(identifier))))))
(statement
(expression
(function_call
(built_in_function
(expression
(identifier)))))))))))))
(statement
(expression
(value
(function
(parameter
(identifier)
(type))
(parameter
(identifier)
(type))
(block
(statement
(expression
(function_call
(built_in_function
(expression
(identifier))))))
(statement
(expression
(function_call
(built_in_function
(expression
(identifier))))))))))))
================================================================================
Complex Function Call
@ -110,34 +112,27 @@ Complex Function Call
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(function_call
(identifier)
(expression
(value
(string)))
(expression
(value
(integer)))
(expression
(value
(map
(block
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(integer))))))
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(integer)))))))))))))))
(statement
(expression
(function_call
(expression
(identifier))
(expression
(value
(string)))
(expression
(value
(integer)))
(expression
(value
(map
(identifier)
(statement
(expression
(value
(integer))))
(identifier)
(statement
(expression
(value
(integer)))))))))))

25
tree-sitter-dust/corpus/identifiers.txt
View File

@ -1,21 +1,20 @@
==================
================================================================================
Simple Identifiers
==================
================================================================================
x
_y
__xyz__
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(identifier)))
(statement
(expression
(identifier)))
(statement
(expression
(identifier)))))
(statement
(expression
(identifier)))
(statement
(expression
(identifier)))
(statement
(expression
(identifier))))

419
tree-sitter-dust/corpus/if_else.txt
View File

@ -1,153 +1,151 @@
==================
================================================================================
Simple If
==================
================================================================================
if true { "True" }
---
--------------------------------------------------------------------------------
(root
(block
(statement
(if_else
(if
(expression
(value
(boolean)))
(block
(statement
(expression
(value
(string))))))))))
(statement
(if_else
(if
(expression
(value
(boolean)))
(block
(statement
(expression
(value
(string)))))))))
==================
================================================================================
Complex If
==================
================================================================================
if 1 == 1 && 2 == 2 && 3 == 3 "True"
if 1 == 1 && 2 == 2 && 3 == 3 { "True" }
---
--------------------------------------------------------------------------------
(root
(block
(statement
(if_else
(if
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(value
(integer)))))))))))))
(block
(statement
(expression
(value
(string))))))))))
(statement
(if_else
(if
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(value
(integer)))))))))))))
(block
(statement
(expression
(value
(string)))))))))
==================
================================================================================
Nested If
==================
================================================================================
if true
if 42 == 12
if true {
if 42 == 12 {
'hiya'
else
} else {
'bye'
}
}
---
--------------------------------------------------------------------------------
(root
(block
(statement
(if_else
(if
(expression
(value
(boolean)))
(block
(statement
(if_else
(if
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(value
(integer)))))
(block
(statement
(expression
(value
(string))))))
(else
(block
(statement
(expression
(value
(string))))))))))))))
(statement
(if_else
(if
(expression
(value
(boolean)))
(block
(statement
(if_else
(if
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(value
(integer)))))
(block
(statement
(expression
(value
(string))))))
(else
(block
(statement
(expression
(value
(string)))))))))))))
==================
================================================================================
If Else
==================
================================================================================
if false "True" else "False"
if false { "True" } else { "False" }
---
--------------------------------------------------------------------------------
(root
(block
(statement
(if_else
(if
(expression
(value
(boolean)))
(block
(statement
(expression
(value
(string))))))
(else
(block
(statement
(expression
(value
(string))))))))))
(statement
(if_else
(if
(expression
(value
(boolean)))
(block
(statement
(expression
(value
(string))))))
(else
(block
(statement
(expression
(value
(string)))))))))
==================
================================================================================
If Else If
==================
================================================================================
if 1 == 1 {
"math is fun"
@ -155,104 +153,103 @@ if 1 == 1 {
"math is broken"
}
---
--------------------------------------------------------------------------------
(root
(block
(statement
(if_else
(if
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(value
(integer)))))
(block
(statement
(expression
(value
(string))))))
(else_if
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(value
(integer)))))
(block
(statement
(expression
(value
(string))))))))))
(statement
(if_else
(if
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(value
(integer)))))
(block
(statement
(expression
(value
(string))))))
(else_if
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(value
(integer)))))
(block
(statement
(expression
(value
(string)))))))))
==================
================================================================================
If Else Else If Else
==================
================================================================================
if false
if false {
"no"
else if false
} else if false {
"no"
else if 1 + 1 == 9
} else if 1 + 1 == 9 {
"not the answer"
else
} else {
"42"
}
---
--------------------------------------------------------------------------------
(root
(block
(statement
(if_else
(if
(expression
(value
(boolean)))
(block
(statement
(expression
(value
(string))))))
(else_if
(expression
(value
(boolean)))
(block
(statement
(expression
(value
(string))))))
(else_if
(expression
(logic
(expression
(math
(expression
(value
(integer)))
(math_operator)
(expression
(value
(integer)))))
(logic_operator)
(expression
(value
(integer)))))
(block
(statement
(expression
(value
(string))))))
(else
(block
(statement
(expression
(value
(string))))))))))
(statement
(if_else
(if
(expression
(value
(boolean)))
(block
(statement
(expression
(value
(string))))))
(else_if
(expression
(value
(boolean)))
(block
(statement
(expression
(value
(string))))))
(else_if
(expression
(logic
(expression
(math
(expression
(value
(integer)))
(math_operator)
(expression
(value
(integer)))))
(logic_operator)
(expression
(value
(integer)))))
(block
(statement
(expression
(value
(string))))))
(else
(block
(statement
(expression
(value
(string)))))))))

142
tree-sitter-dust/corpus/index.txt
View File

@ -1,6 +1,6 @@
==================
================================================================================
Simple Indexes
==================
================================================================================
dust_data:1:name
@ -8,87 +8,107 @@ creature:total_clams
foobar:1:42
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(index
(expression
(index
(expression
(identifier))
(expression
(value
(integer)))))
(expression
(identifier)))))
(statement
(expression
(index
(expression
(identifier))
(expression
(identifier)))))
(statement
(statement
(expression
(index
(expression
(index
(expression
(index
(expression
(identifier))
(expression
(value
(integer)))))
(identifier))
(expression
(value
(integer))))))))
(integer)))))
(expression
(identifier)))))
(statement
(expression
(index
(expression
(identifier))
(expression
(identifier)))))
(statement
(expression
(index
(expression
(index
(expression
(identifier))
(expression
(value
(integer)))))
(expression
(value
(integer)))))))
==================
================================================================================
Nested Indexes
==================
================================================================================
[['answers' 'foobar'], 42, 666]:0:1:0..2
---
--------------------------------------------------------------------------------
(root
(block
(statement
(statement
(expression
(index
(expression
(index
(expression
(index
(expression
(index
(expression
(value
(list
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(string))))))
(expression
(value
(integer)))
(expression
(value
(integer))))))
(expression
(value
(integer)))))
(value
(list
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(string))))))
(expression
(value
(integer)))
(expression
(value
(integer))))))
(expression
(value
(integer)))))
(expression
(value
(integer)))
(integer)))))
(expression
(value
(integer)))
(expression
(value
(integer)))))))
================================================================================
Function Call Index
================================================================================
x:(y):0
--------------------------------------------------------------------------------
(root
(statement
(expression
(index
(expression
(index
(expression
(value
(integer))))))))
(identifier))
(expression
(identifier))))
(expression
(value
(integer)))))))

72
tree-sitter-dust/corpus/lists.txt
View File

@ -1,50 +1,48 @@
==================
================================================================================
List Declaration
==================
================================================================================
['answer', 42]
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(integer)))))))))
(statement
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(integer))))))))
==================
================================================================================
List Nesting
==================
================================================================================
['answers', [42, [666]]]
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(list
(expression
(value
(integer)))
(expression
(value
(list
(expression
(value
(integer)))))))))))))))
(statement
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(list
(expression
(value
(integer)))
(expression
(value
(list
(expression
(value
(integer))))))))))))))

98
tree-sitter-dust/corpus/maps.txt
View File

@ -7,20 +7,15 @@ map { answer = 42 }
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(value
(map
(block
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(integer)))))))))))))
(statement
(expression
(value
(map
(identifier)
(statement
(expression
(value
(integer)))))))))
================================================================================
Nested Maps
@ -39,55 +34,36 @@ x = map {
--------------------------------------------------------------------------------
(root
(block
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(map
(block
(statement
(assignment
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(map
(identifier)
(statement
(expression
(value
(map
(identifier)
(statement
(expression
(value
(string))))
(identifier)
(assignment_operator)
(statement
(expression
(value
(map
(block
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(string))))))
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(map
(block
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(string))))))))))))))))))))
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(integer)))))))))))))))
(identifier)
(statement
(expression
(value
(string))))))))))))
(identifier)
(statement
(expression
(value
(integer)))))))))))

153
tree-sitter-dust/corpus/operators.txt
View File

@ -1,98 +1,95 @@
==================
================================================================================
\==
==================
================================================================================
3 == 1 + 1 + 1
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(math
(expression
(math
(expression
(value
(integer)))
(math_operator)
(expression
(value
(integer)))))
(math_operator)
(expression
(value
(integer))))))))))
(statement
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(math
(expression
(math
(expression
(value
(integer)))
(math_operator)
(expression
(value
(integer)))))
(math_operator)
(expression
(value
(integer)))))))))
==================
================================================================================
&&
==================
================================================================================
4 + 2 == 42 && true
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(logic
(expression
(math
(expression
(value
(integer)))
(math_operator)
(expression
(value
(integer)))))
(logic_operator)
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(value
(boolean))))))))))
(statement
(expression
(logic
(expression
(math
(expression
(value
(integer)))
(math_operator)
(expression
(value
(integer)))))
(logic_operator)
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(value
(boolean)))))))))
==================
================================================================================
\||
==================
================================================================================
4 + 2 == 42 || true
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(logic
(expression
(math
(expression
(value
(integer)))
(math_operator)
(expression
(value
(integer)))))
(logic_operator)
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(value
(boolean))))))))))
(statement
(expression
(logic
(expression
(math
(expression
(value
(integer)))
(math_operator)
(expression
(value
(integer)))))
(logic_operator)
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(value
(boolean)))))))))

73
tree-sitter-dust/corpus/reduce.txt
View File

@ -1,73 +0,0 @@
==================
Simple Reduce Loop
==================
reduce i to acc in [1, 2, 3] {
acc += i
}
---
(root
(block
(statement
(reduce
(identifier)
(identifier)
(expression
(value
(list
(expression
(value
(integer)))
(expression
(value
(integer)))
(expression
(value
(integer))))))
(block
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(identifier))))))))))
==================
Nested Reduce Loop
==================
reduce i to acc in ["one", "two", "three"] {
acc += i
}
---
(root
(block
(statement
(reduce
(identifier)
(identifier)
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(string)))
(expression
(value
(string))))))
(block
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(identifier))))))))))

73
tree-sitter-dust/corpus/remove.txt
View File

@ -1,73 +0,0 @@
==================
Simple Remove
==================
remove i from [1, 2, 3] {
i <= 2
}
---
(root
(block
(statement
(remove
(identifier)
(expression
(value
(list
(expression
(value
(integer)))
(expression
(value
(integer)))
(expression
(value
(integer))))))
(block
(statement
(expression
(logic
(expression
(identifier))
(logic_operator)
(expression
(value
(integer)))))))))))
==================
Nested Remove
==================
remove i from big_list {
remove j from i {
j != 42
}
}
---
(root
(block
(statement
(remove
(identifier)
(expression
(identifier))
(block
(statement
(remove
(identifier)
(expression
(identifier))
(block
(statement
(expression
(logic
(expression
(identifier))
(logic_operator)
(expression
(value
(integer))))))))))))))

200
tree-sitter-dust/corpus/statements.txt
View File

@ -1,58 +1,56 @@
==================
================================================================================
Simple Statements
==================
================================================================================
1
"one";
x
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(value
(integer))))
(statement
(expression
(value
(string))))
(statement
(expression
(identifier)))))
(statement
(expression
(value
(integer))))
(statement
(expression
(value
(string))))
(statement
(expression
(identifier))))
==================
================================================================================
Simple Assignment
==================
================================================================================
x = 1;
y = "one"
---
--------------------------------------------------------------------------------
(root
(block
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(integer))))))
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(string))))))))
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(integer))))))
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(string)))))))
==================
================================================================================
Complex Assignment
==================
================================================================================
x = if 1 + 1 == 2 {
'yo'
@ -60,77 +58,75 @@ x = if 1 + 1 == 2 {
'no'
}
---
--------------------------------------------------------------------------------
(root
(block
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(if_else
(if
(expression
(logic
(expression
(math
(expression
(value
(integer)))
(math_operator)
(expression
(value
(integer)))))
(logic_operator)
(expression
(value
(integer)))))
(block
(statement
(expression
(value
(string))))))
(else
(block
(statement
(expression
(value
(string))))))))))))
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(if_else
(if
(expression
(logic
(expression
(math
(expression
(value
(integer)))
(math_operator)
(expression
(value
(integer)))))
(logic_operator)
(expression
(value
(integer)))))
(block
(statement
(expression
(value
(string))))))
(else
(block
(statement
(expression
(value
(string)))))))))))
==================
================================================================================
Expression Precedence
==================
================================================================================
x = 3 == 1 + 2 + 2
---
--------------------------------------------------------------------------------
(root
(block
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(math
(expression
(math
(expression
(value
(integer)))
(math_operator)
(expression
(value
(integer)))))
(math_operator)
(expression
(value
(integer))))))))))))
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(logic
(expression
(value
(integer)))
(logic_operator)
(expression
(math
(expression
(math
(expression
(value
(integer)))
(math_operator)
(expression
(value
(integer)))))
(math_operator)
(expression
(value
(integer)))))))))))

157
tree-sitter-dust/corpus/tables.txt
View File

@ -1,6 +1,6 @@
==================
================================================================================
Table Declaration
==================
================================================================================
table |messages numbers| [
['hiya' 42]
@ -8,101 +8,98 @@ table |messages numbers| [
['bar' 99.99]
]
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(value
(table
(identifier_list
(identifier)
(identifier))
(expression
(value
(list
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(integer))))))
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(integer))))))
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(float)))))))))))))))
(statement
(expression
(value
(table
(identifier_list
(identifier)
(identifier))
(expression
(value
(list
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(integer))))))
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(integer))))))
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(float))))))))))))))
==================
================================================================================
Table Access
==================
================================================================================
select |number| from foobar {
text == 'answer'
}
---
--------------------------------------------------------------------------------
(root
(block
(statement
(select
(identifier_list
(identifier))
(expression
(identifier))
(block
(statement
(expression
(logic
(expression
(identifier))
(logic_operator)
(expression
(value
(string)))))))))))
(statement
(select
(identifier_list
(identifier))
(expression
(identifier))
(block
(statement
(expression
(logic
(expression
(identifier))
(logic_operator)
(expression
(value
(string))))))))))
==================
================================================================================
Table Insert
==================
================================================================================
insert into foobar [
['bob was here', 0]
]
---
--------------------------------------------------------------------------------
(root
(block
(statement
(insert
(identifier)
(expression
(value
(list
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(integer)))))))))))))
(statement
(insert
(identifier)
(expression
(value
(list
(expression
(value
(list
(expression
(value
(string)))
(expression
(value
(integer))))))))))))

88
tree-sitter-dust/corpus/transform.txt
View File

@ -1,88 +0,0 @@
================================================================================
Transform Loop
================================================================================
transform i in [1, 2, 3] {
(output i)
}
--------------------------------------------------------------------------------
(root
(block
(statement
(transform
(identifier)
(expression
(value
(list
(expression
(value
(integer)))
(expression
(value
(integer)))
(expression
(value
(integer))))))
(block
(statement
(expression
(function_call
(built_in_function
(expression
(identifier)))))))))))
================================================================================
Nested Transform Loop
================================================================================
transform i in [['one'] ['two'] ['three']] {
transform j in i {
j += 'foobar'
}
}
--------------------------------------------------------------------------------
(root
(block
(statement
(transform
(identifier)
(expression
(value
(list
(expression
(value
(list
(expression
(value
(string))))))
(expression
(value
(list
(expression
(value
(string))))))
(expression
(value
(list
(expression
(value
(string)))))))))
(block
(statement
(transform
(identifier)
(expression
(identifier))
(block
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(string))))))))))))))

117
tree-sitter-dust/corpus/values.txt
View File

@ -1,82 +1,79 @@
==================
================================================================================
Booleans
==================
================================================================================
true
false
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(value
(boolean))))
(statement
(expression
(value
(boolean))))))
(statement
(expression
(value
(boolean))))
(statement
(expression
(value
(boolean)))))
==================
================================================================================
Integers
==================
================================================================================
1 2 3
456 7
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(value
(integer))))
(statement
(expression
(value
(integer))))
(statement
(expression
(value
(integer))))
(statement
(expression
(value
(integer))))
(statement
(expression
(value
(integer))))))
(statement
(expression
(value
(integer))))
(statement
(expression
(value
(integer))))
(statement
(expression
(value
(integer))))
(statement
(expression
(value
(integer))))
(statement
(expression
(value
(integer)))))
==================
================================================================================
Strings
==================
================================================================================
"one" 'two' "three" `four` 'five'
---
--------------------------------------------------------------------------------
(root
(block
(statement
(expression
(value
(string))))
(statement
(expression
(value
(string))))
(statement
(expression
(value
(string))))
(statement
(expression
(value
(string))))
(statement
(expression
(value
(string))))))
(statement
(expression
(value
(string))))
(statement
(expression
(value
(string))))
(statement
(expression
(value
(string))))
(statement
(expression
(value
(string))))
(statement
(expression
(value
(string)))))

78
tree-sitter-dust/corpus/while.txt
View File

@ -9,20 +9,19 @@ while true {
--------------------------------------------------------------------------------
(root
(block
(statement
(while
(expression
(value
(boolean)))
(block
(statement
(expression
(function_call
(built_in_function
(expression
(value
(string))))))))))))
(statement
(while
(expression
(value
(boolean)))
(block
(statement
(expression
(function_call
(built_in_function
(expression
(value
(string)))))))))))
================================================================================
Nested While Loop
@ -37,29 +36,28 @@ while (true) {
--------------------------------------------------------------------------------
(root
(block
(statement
(while
(expression
(value
(boolean)))
(block
(statement
(while
(expression
(logic
(expression
(identifier))
(logic_operator)
(expression
(value
(integer)))))
(block
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(integer))))))))))))))
(statement
(while
(expression
(value
(boolean)))
(block
(statement
(while
(expression
(logic
(expression
(identifier))
(logic_operator)
(expression
(value
(integer)))))
(block
(statement
(assignment
(identifier)
(assignment_operator)
(statement
(expression
(value
(integer)))))))))))))

41
tree-sitter-dust/corpus/yield.txt Normal file
View File

@ -0,0 +1,41 @@
================================================================================
Simple Yield
================================================================================
1 -> (output)
--------------------------------------------------------------------------------
(root
(statement
(expression
(yield
(expression
(value
(integer)))
(built_in_function)))))
================================================================================
Yield Chain
================================================================================
x -> (foo) -> (bar) -> (abc)
--------------------------------------------------------------------------------
(root
(statement
(expression
(yield
(expression
(yield
(expression
(yield
(expression
(identifier))
(expression
(identifier))))
(expression
(identifier))))
(expression
(identifier))))))

173
tree-sitter-dust/grammar.js
View File

@ -3,57 +3,57 @@ module.exports = grammar({
word: $ => $.identifier,
extras: $ => [ /\s/, $.comment ],
conflicts: $ => [
[$.block],
],
extras: $ => [ /\s/, $._comment ],
rules: {
root: $ => $.block,
root: $ => prec(1, repeat1($.statement)),
comment: $ => /[#][^#\n]*[#|\n]/,
_comment: $ => /[#][^#\n]*[#|\n]/,
block: $ => prec.right(choice(
block: $ => seq(
optional('async'),
'{',
repeat1($.statement),
seq('{', repeat1($.statement), '}'),
)),
'}',
),
statement: $ => prec.right(seq(
statement: $ => prec.left(seq(
choice(
$.assignment,
$.await,
$.block,
$.expression,
$.filter,
$.find,
$.for,
$.if_else,
$.index_assignment,
$.insert,
$.match,
$.reduce,
$.remove,
$.return,
$.select,
$.transform,
$.use,
$.while,
),
optional(';'),
)),
expression: $ => prec.right(choice(
$._expression_kind,
seq('(', $._expression_kind, ')'),
)),
_expression_kind: $ => prec.right(1, choice(
_expression_kind: $ => prec.right(choice(
$.function_call,
$.identifier,
$.index,
$.logic,
$.math,
$.value,
$.yield,
)),
_expression_list: $ => repeat1(prec.right(seq($.expression, optional(',')))),
_expression_list: $ => repeat1(prec.right(seq(
$.expression,
optional(','),
))),
identifier: $ => /[_a-zA-Z]+[_a-zA-Z0-9]?/,
@ -113,7 +113,7 @@ module.exports = grammar({
'}',
),
index: $ => prec.left(seq(
index: $ => prec.left(1, seq(
$.expression,
':',
$.expression,
@ -155,16 +155,23 @@ module.exports = grammar({
),
assignment: $ => seq(
$.identifier,
field('identifier', $.identifier),
optional(field('type', $.type)),
field('assignment_operator', $.assignment_operator),
field('statement', $.statement),
),
index_assignment: $ => seq(
$.index,
$.assignment_operator,
$.statement,
),
assignment_operator: $ => choice(
assignment_operator: $ => prec.right(choice(
"=",
"+=",
"-=",
),
)),
if_else: $ => prec.right(seq(
$.if,
@ -206,50 +213,10 @@ module.exports = grammar({
),
for: $ => seq(
'for',
$.identifier,
'in',
$.expression,
$.block,
),
transform: $ => seq(
'transform',
$.identifier,
'in',
$.expression,
$.block,
),
filter: $ => seq(
'filter',
field('count', optional($.expression)),
field('statement_id', $.identifier),
'in',
field('collection', $.expression),
field('predicate', $.block),
),
find: $ => seq(
'find',
$.identifier,
'in',
$.expression,
$.block,
),
remove: $ => seq(
'remove',
$.identifier,
'from',
$.expression,
$.block,
),
reduce: $ => seq(
'reduce',
$.identifier,
'to',
choice(
'for',
'async for',
),
$.identifier,
'in',
$.expression,
@ -270,7 +237,7 @@ module.exports = grammar({
$.identifier,
$.expression,
)),
identifier_list: $ => prec.right(choice(
seq(
'|',
@ -285,19 +252,56 @@ module.exports = grammar({
$.expression,
)),
return: $ => seq(
'return',
$.expression,
),
use: $ => seq(
'use',
$.string,
),
type: $ => seq(
'<',
choice(
'any',
'bool',
'fn',
'int',
'list',
'map',
'str',
'table',
),
'>',
),
function: $ => seq(
field('parameters', optional($.identifier_list)),
'=>',
field('body', $.block),
'|',
repeat($.parameter),
'|',
optional($.type),
$.block,
),
function_call: $ => choice(
$.built_in_function,
$._context_defined_function,
),
_context_defined_function: $ => prec.right(seq(
parameter: $ => seq(
$.identifier,
$.type,
optional(','),
),
function_call: $ => prec.right(1, seq(
'(',
choice(
$.built_in_function,
$._context_defined_function,
),
')',
)),
_context_defined_function: $ => prec.right(1, seq(
$.expression,
optional($._expression_list),
)),
@ -306,10 +310,22 @@ module.exports = grammar({
optional($._expression_list),
)),
yield: $ => prec.left(seq(
$.expression,
'->',
'(',
choice(
$.built_in_function,
$._context_defined_function,
),
')',
)),
_built_in_function_name: $ => choice(
// General
'assert',
'assert_equal',
'context',
'download',
'help',
'length',
@ -352,4 +368,3 @@ module.exports = grammar({
'reverse',
),
}
});