jeff/dust
1
0
Fork 0
Code Issues

Edit README; Fix a test; Try tracing the compiler and VM

Browse Source
This commit is contained in:
Jeff 2024-12-26 14:29:04 -05:00
parent dc3cc13b12
commit 950c36d601
5 changed files with 72 additions and 154 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

172
README.md
View File

@ -48,16 +48,6 @@ fn fib (n: int) -> int {
write_line(fib(25))
```
Dust is an interpreted language that is compiled to bytecode by a hand-written lexer and parser. It
uses a custom multi-threaded register-based virtual machine with separate-thread garbage collection.
Competing with the runtime performance of Rust or C++ *is not* a goal. But competing with the
approachability and simplicity of those languages *is* a goal. Dust *does* intend to be faster than
Python, Ruby and NodeJS while also offering a superior development experience and more reliable code
due to its static typing. Dust's development approach is informed by some books[^1] and academic
research[^4] as well as practical insight from papers[^2] written by language authors. See the
[Inspiration](README#Inspiration) section for more information or keep reading to learn about Dust's
features.
## Goals
This project's goal is to deliver a language that not only *works* but that offers genunine value
@ -94,56 +84,6 @@ first sentence, Dust's general aspirations are to be **fast**, **safe** and **ea
- **Low Resource Usage** Despite its performance, Dust's use of memory and CPU power should be
conservative and predictable enough to accomodate a wide range of devices.
These are the project's general design goals. There are many more implementation goals. Among them
are:
- Effortless Concurrency: Dust should offer an excellent experience for writing multi-threaded
programs. The language's native functions should offer an API for spawning threads, sending
messages and waiting for results. When using these features, Dust should be much faster than any
single-threaded language. However, Dust should be fast even when running on a single thread.
Single-threaded performce is the best predictor of multi-threaded performance so continuing to
optimize how each thread executes instructions, accesses memory and moves pointers is the best
way to ensure that Dust is fast in all scenarios.
- Embeddability: The library should be easy to use so that Dust can be built into other
applications. Dust should compile to WebAssembly and offer examples of how to use it in a web
application. The user should be able to query the VM for information about the program's state
and control the program's execution. It should be possible to view and modify the value of a
variable and inspect the call stack.
- Data Fluency: Dust's value type should support conversion to and from arbitrary data in formats
like JSON, YAML, TOML and CSV. Pulling data into a Dust program should be easy, with built-in
functions offering conversion for the most widely used formats.
- Portability: Dust should run on as many architectures and operating systems as possible. Using
fewer dependencies and avoiding platform-specific code will help Dust achieve this goal. The
Dust library should be available as a WebAssembly module.
- Developer Experience: Dust should be fun and easy to use. That implies easy installation and the
availability of tutorials and how-to guides. The CLI should be predictable and feature-rich,
with features that make it easy to write and debug Dust code like formatting, bytecode
disassembly and logging.
- Advanced Type System: Dust should implement composite types, aliases and generics. The type
system should use a descriptive syntax that is easy to understand. Dust's type system should be
static, meaning that types are checked before a program reaches the VM. Dust is not a
graduallly typed language, its VM is and should remain type-agnostic.
- Thorough Testing: Primarily, the output of Dust's compiler and VM should be tested with programs
that cover all of the language's features. The tests should be actively maintained and should be
changed frequently to reflect a growing project that is constantly discovering new optimizations
and opportunities for improvement.
## Project Status
This project is maintained by a single developer. For now, its primary home is on a private git
server. The GitHub mirror is updated automatically and should carry the latest branches. There are
no other contributors at this time but the project is open to feedback and should eventually accept
contributions.
For now, both the library API and the implementation details are freely changed and the CLI has not
been published. Dust is both an ambitious project and a continuous experiment in language design.
Features may be redesigned and reimplemented at will when they do not meet the project's performance
or usability goals. This approach maximizes the development experience as a learning opportunity and
enforces a high standard of quality but slows down the process of delivering features to users.
Eventually, Dust will reach a stable release and will be ready for general use. As the project
approaches this milestone, the experimental nature of the project will be reduced and a replaced
with a focus on stability and improvement.
## Language Overview
This is a quick overview of Dust's syntax features. It skips over the aspects that are familiar to
@ -167,57 +107,6 @@ So while the syntax is by no means compatible, it is superficially similar, even
syntax highlighting for Rust code works well with Dust code. This is not a design goal but a happy
coincidence.
### Semicolons
Dust borrowed Rust's approach to semicolons and their effect on evaluation and relaxed the rules to
accomated different styles of coding. Rust, isn't designed for command lines or REPLs but Dust could
be well-suited to those applications. Dust needs to work in a source file or in an ad-hoc one-liner
sent to the CLI. Thus, semicolons are optional in most cases.
There are two things you need to know about semicolons in Dust:
- Semicolons suppress the value of whatever they follow. The preceding statement or expression will
have the type `none` and will not evaluate to a value.
- If a semicolon does not change how the program runs, it is optional.
This example shows three statements with semicolons. The compiler knows that a `let` statement
cannot produce a value and will always have the type `none`. Thanks to static typing, it also knows
that the `write_line` function has no return value so the function call also has the type `none`.
Therefore, these semicolons are optional.
```rust
let a = 40;
let b = 2;
write_line("The answer is ", a + b);
```
Removing the semicolons does not alter the execution pattern or the return value.
```rust
let x = 10
let y = 3
write_line("The remainder is ", x % y)
```
The next example produces a compiler error because the `if` block returns a value of type `int` but
the `else` block does not return a value at all. Dust does not allow branches of the same `if/else`
statement to have different types. In this case, adding a semicolon after the `777` expression fixes
the error by supressing the value.
```rust
// !!! Compile Error !!!
let input = read_line()
let reward = if input == "42" {
write_line("You got it! Here's your reward.")
777 // <- We need a semicolon here
} else {
write_line(input, " is not the answer.")
}
```
### Statements and Expressions
Dust is composed of statements and expressions. If a statement ends in an expression without a
@ -270,13 +159,54 @@ model is more accomodating without sacrificing expressiveness. In Rust, semicolo
and *meaningful*, which provides excellent consistency but lacks flexibility. In JavaScript,
semicolons are *required* and *meaningless*, which is a source of confusion for many developers.
### Control Flow
Dust borrowed Rust's approach to semicolons and their effect on evaluation and relaxed the rules to
accomated different styles of coding. Rust isn't designed for command lines or REPLs but Dust is
well-suited to those applications. Dust needs to work in a source file or in an ad-hoc one-liner
sent to the CLI. Thus, semicolons are optional in most cases.
-- TODO --
There are two things you need to know about semicolons in Dust:
### Functions
- Semicolons suppress the value of whatever they follow. The preceding statement or expression will
have the type `none` and will not evaluate to a value.
- If a semicolon does not change how the program runs, it is optional.
-- TODO --
This example shows three statements with semicolons. The compiler knows that a `let` statement
cannot produce a value and will always have the type `none`. Thanks to static typing, it also knows
that the `write_line` function has no return value so the function call also has the type `none`.
Therefore, these semicolons are optional.
```rust
let a = 40;
let b = 2;
write_line("The answer is ", a + b);
```
Removing the semicolons does not alter the execution pattern or the return value.
```rust
let x = 10
let y = 3
write_line("The remainder is ", x % y)
```
The next example produces a compiler error because the `if` block returns a value of type `int` but
the `else` block does not return a value at all. Dust does not allow branches of the same `if/else`
statement to have different types. In this case, adding a semicolon after the `777` expression fixes
the error by supressing the value.
```rust
// !!! Compile Error !!!
let input = read_line()
let reward = if input == "42" {
write_line("You got it! Here's your reward.")
777 // <- We need a semicolon here
} else {
write_line(input, " is not the answer.")
}
```
#### Type System
@ -296,14 +226,6 @@ Dust *does* have a `none` type, which should not be confused for being `null`-li
"unit" type in Rust, `none` exists as a type but not as a value. It indicates the lack of a value
from a function, expression or statement. A variable cannot be assigned to `none`.
#### Immutability by Default
TODO
#### Memory Safety
TODO
### Basic Values
Dust supports the following basic values:
@ -324,10 +246,6 @@ singular values. Shorter strings are stored on the stack while longer strings ar
Dust offers built-in native functions that can manipulate strings by accessing their bytes or
reading them as a sequence of characters.
### Composite Values
TODO
## Previous Implementations
Dust has gone through several iterations, each with its own design choices. It was originally

3
dust-cli/src/main.rs
View File

@ -309,9 +309,6 @@ fn main() {
input,
}) = mode
{
let run_span = span!(Level::TRACE, "CLI Run Mode");
let _run_guard = run_span.enter();
let (source, file_name) = get_source_and_file_name(input);
let lexer = Lexer::new(&source);
let mut compiler = match Compiler::new(lexer) {

21
dust-lang/src/compiler/mod.rs
View File

@ -25,7 +25,7 @@ mod type_checks;
pub use error::CompileError;
use parse_rule::{ParseRule, Precedence};
use tracing::{debug, info};
use tracing::{debug, info, span, Level};
use type_checks::{check_math_type, check_math_types};
use std::mem::replace;
@ -135,12 +135,6 @@ impl<'src> Compiler<'src> {
pub fn new(mut lexer: Lexer<'src>) -> Result<Self, CompileError> {
let (current_token, current_position) = lexer.next_token()?;
info!(
"Begin chunk with {} at {}",
current_token.to_string(),
current_position.to_string()
);
Ok(Compiler {
function_name: None,
r#type: FunctionType {
@ -172,8 +166,6 @@ impl<'src> Compiler<'src> {
/// will allow [`Compiler::function_name`] to be both the name used for recursive calls and the
/// name of the function when it is compiled. The name can later be seen in the VM's call stack.
pub fn finish(self, name: Option<impl Into<DustString>>) -> Chunk {
info!("End chunk");
let (instructions, positions): (SmallVec<[Instruction; 32]>, SmallVec<[Span; 32]>) = self
.instructions
.into_iter()
@ -205,6 +197,15 @@ impl<'src> Compiler<'src> {
/// [`CompileError`] if any are found. After calling this function, check its return value for
/// an error, then call [`Compiler::finish`] to get the compiled chunk.
pub fn compile(&mut self) -> Result<(), CompileError> {
let span = span!(Level::INFO, "Compiling");
let _enter = span.enter();
info!(
"Begin chunk with {} at {}",
self.current_token.to_string(),
self.current_position.to_string()
);
loop {
self.parse(Precedence::None)?;
@ -219,6 +220,8 @@ impl<'src> Compiler<'src> {
}
}
info!("End chunk");
Ok(())
}

4
dust-lang/src/vm/mod.rs
View File

@ -16,6 +16,7 @@ pub use error::VmError;
pub use record::Record;
pub use run_action::RunAction;
pub use thread::{Thread, ThreadSignal};
use tracing::{span, Level};
use crate::{compile, Chunk, DustError, Value};
@ -40,6 +41,9 @@ impl Vm {
}
pub fn run(mut self) -> Option<Value> {
let span = span!(Level::INFO, "Running");
let _enter = span.enter();
if self.threads.len() == 1 {
return self.threads[0].run();
}

26
dust-lang/tests/math/add_assign.rs
View File

@ -1,4 +1,5 @@
use dust_lang::*;
use smallvec::smallvec;
#[test]
fn add_assign() {
@ -13,23 +14,18 @@ fn add_assign() {
value_parameters: None,
return_type: Type::Integer,
},
vec![
(Instruction::load_constant(0, 0, false), Span(12, 13)),
(
Instruction::add(0, Argument::Register(0), Argument::Constant(2)),
Span(17, 19)
),
(Instruction::get_local(1, 0), Span(23, 24)),
(Instruction::r#return(true), Span(24, 24))
smallvec![
Instruction::load_constant(0, 0, false),
Instruction::add(0, Argument::Register(0), Argument::Constant(2)),
Instruction::get_local(1, 0),
Instruction::r#return(true)
],
vec![
ConcreteValue::Integer(1),
ConcreteValue::string("a"),
ConcreteValue::Integer(2)
],
vec![Local::new(1, 0, true, Scope::default())]
smallvec![Span(12, 13), Span(17, 19), Span(23, 24), Span(24, 24)],
smallvec![Value::integer(1), Value::string("a"), Value::integer(2)],
smallvec![Local::new(1, 0, true, Scope::default())],
vec![]
))
);
assert_eq!(run(source), Ok(Some(ConcreteValue::Integer(3))));
assert_eq!(run(source), Ok(Some(Value::integer(3))));
}