1304e04808
[lexer] Bring over the REPL from the parser
2018-08-26 13:45:52 -07:00
a23b917785
[lexer] Lex bools!
...
Extend the lexer to support Scheme bools like this: #t, #f, #true, #false.
Add some positive tests for this too.
2018-08-26 13:40:27 -07:00
b0b4699476
[lexer] Rewrite the Lexer!
...
Implement structs for each state in the lexer. The State objects are responsible
for determining how to handle a character and emit a Result indicating to the
driver (the Lexer struct) how to proceed.
2018-08-26 11:47:18 -07:00
4fa07ceec7
[lexer] Add Error::msg()
2018-08-23 20:19:16 -07:00
8475720a71
Add accessor for Lex::value
2018-08-20 15:21:20 -07:00
281c29ee26
[lexer] Expose token field via method; Clone and Copy Tokens
2018-08-19 22:30:19 -07:00
cc43ffd135
[lexer] Lexer emits Lexes instead of Tokens
2017-06-26 21:54:57 -07:00
5f3770914f
[lexer] Basic handling of input offsets
2017-06-26 21:54:05 -07:00
2a7626c75f
Add a type def for lexer Result
2017-06-18 16:50:14 -07:00
d994316392
Character checks for identifier initial and subsequent
2017-05-13 17:21:23 -07:00
237dca4b4b
Add some character class methods to a Lexable trait for char
2017-05-13 15:37:01 -07:00
1dfdc001b3
Add an error class
2017-05-13 15:26:41 -07:00
b5f76deb98
Add some single token tests for the tokens we have
2017-05-02 21:45:10 -07:00
28e5814101
Check for EOF (aka input.next() -> None) and emit what we have before finishing forever
2017-05-02 21:44:57 -07:00
5fe10fe002
Simplify the resume check -- just use an if
2017-05-02 21:44:01 -07:00
c5b769ff45
A peekable lexer
...
Use the chars() iterator on &str, with the Peekable wrapper on Iterators, to iterate the input, rather than needing to hold the whole input and do iteration by indexes.
2017-04-30 17:46:42 -07:00
a4282e7760
Blow away the lexer code =o
2017-04-30 16:32:31 -07:00
31c68034a8
Fix a good portion of the build errors in sibillexer (but not all...)
2017-04-30 16:00:51 -07:00
027daf692b
lexer::number uses sibiltypes::Object
2017-04-30 15:57:16 -07:00
21263fb2ed
Lexer doesn't have a named_char module anymore
2017-04-30 15:57:02 -07:00
a88bf20a23
Derive Eq on lexer::number::Sign
2017-04-30 15:56:49 -07:00
28ba98582a
More compact formatting
2017-04-30 15:56:32 -07:00
9183e05891
Add FromChar trait to lexer::char
2017-04-30 15:56:21 -07:00
3ffb694ce1
Move Exact to types::number
2017-04-30 15:55:50 -07:00
fd5df91e27
[lexer,types] Move named_char to types as types/char/names.rs
...
Move char.rs to be char/mod.rs
2017-04-23 10:41:47 -07:00
5cb66d932f
[lexer] Clean up named_char module
2017-04-23 10:02:39 -07:00
a5f88c221d
Use Objects for Token values
2017-04-22 15:21:13 -07:00
9f1500c1b6
Improve logging ever so slightly
2017-04-15 18:12:24 -07:00
ddf13f9bae
Make Lex less mutable
2017-04-15 18:12:24 -07:00
f9d9a08455
Add a failing test that the lexer finds rational numbers
2017-04-15 10:35:13 -07:00
4f11be1316
Remove a println!
2017-04-15 10:17:50 -07:00
50baf2c118
Make all the lexer tests pass again
...
Try to make exact numbers if possible...
2017-04-15 09:46:21 -07:00
fc947280ae
Move the lexer to its own sibillexer module
...
Lots of failing tests right now, unfortunately. :(
2017-04-15 09:37:12 -07:00
