176686b3a7
[lexer] Let states handle their own creation in from_char
2018-09-03 16:06:52 -07:00
eabc8f98e8
[lexer] Add number::Sign state
2018-09-03 15:49:13 -07:00
663ae3a9f1
[lexer] move char::is_hash() to chars::Lexable
2018-09-03 15:48:48 -07:00
dc8f5a7686
[lexer] Replace String error messages with Error type
2018-09-03 15:30:48 -07:00
def35966eb
[lexer] Re-kajigger the states for numbers
2018-09-03 15:19:28 -07:00
b759ee4c57
[lexer] Add Error::invalid_char constructor
2018-09-03 15:18:15 -07:00
469929fb8f
[lexer] Make a states::number module
...
There will be several number-specific states.
2018-09-03 14:05:57 -07:00
14ea07e441
[lexer] Move Bool state to its own file
2018-09-03 12:27:09 -07:00
ebee8c8646
[lexer] Add Num token type, which only takes ints for now...
2018-09-02 17:24:07 -07:00
0f18569292
[lexer] Add (failing) test for integers
2018-09-02 17:23:48 -07:00
04f2eb0937
[lexer] Remove empty Lexer impl
2018-09-02 17:23:32 -07:00
757f943fff
[lexer] Oops forgot a return type on Hash::new()
2018-09-02 14:05:56 -07:00
7a6c2b91d1
[lexer] Bit of code cleanup in Lexer
2018-09-02 14:05:35 -07:00
f35fe5fd08
[lexer] Implement Hash::new()
2018-09-02 13:50:33 -07:00
aa4de7d4bd
[lexer] Laying groundwork for lexing numbers...
2018-09-01 22:25:40 -07:00
7b6259977f
[lexer] Add fail() constructor to StateResult
2018-09-01 22:21:21 -07:00
d69c3dbc31
[lexer] Obey Rust book recommendation about order of leading lines
2018-09-01 22:21:05 -07:00
e61dc0b6a4
[lexer] Add failing test for issue #12
2018-08-27 06:56:48 -07:00
1304e04808
[lexer] Bring over the REPL from the parser
2018-08-26 13:45:52 -07:00
a23b917785
[lexer] Lex bools!
...
Extend the lexer to support Scheme bools like this: #t, #f, #true, #false.
Add some positive tests for this too.
2018-08-26 13:40:27 -07:00
b0b4699476
[lexer] Rewrite the Lexer!
...
Implement structs for each state in the lexer. The State objects are responsible
for determining how to handle a character and emit a Result indicating to the
driver (the Lexer struct) how to proceed.
2018-08-26 11:47:18 -07:00
4fa07ceec7
[lexer] Add Error::msg()
2018-08-23 20:19:16 -07:00
8475720a71
Add accessor for Lex::value
2018-08-20 15:21:20 -07:00
281c29ee26
[lexer] Expose token field via method; Clone and Copy Tokens
2018-08-19 22:30:19 -07:00
cc43ffd135
[lexer] Lexer emits Lexes instead of Tokens
2017-06-26 21:54:57 -07:00
5f3770914f
[lexer] Basic handling of input offsets
2017-06-26 21:54:05 -07:00
2a7626c75f
Add a type def for lexer Result
2017-06-18 16:50:14 -07:00
d994316392
Character checks for identifier initial and subsequent
2017-05-13 17:21:23 -07:00
237dca4b4b
Add some character class methods to a Lexable trait for char
2017-05-13 15:37:01 -07:00
1dfdc001b3
Add an error class
2017-05-13 15:26:41 -07:00
b5f76deb98
Add some single token tests for the tokens we have
2017-05-02 21:45:10 -07:00
28e5814101
Check for EOF (aka input.next() -> None) and emit what we have before finishing forever
2017-05-02 21:44:57 -07:00
5fe10fe002
Simplify the resume check -- just use an if
2017-05-02 21:44:01 -07:00
c5b769ff45
A peekable lexer
...
Use the chars() iterator on &str, with the Peekable wrapper on Iterators, to iterate the input, rather than needing to hold the whole input and do iteration by indexes.
2017-04-30 17:46:42 -07:00
a4282e7760
Blow away the lexer code =o
2017-04-30 16:32:31 -07:00
31c68034a8
Fix a good portion of the build errors in sibillexer (but not all...)
2017-04-30 16:00:51 -07:00
027daf692b
lexer::number uses sibiltypes::Object
2017-04-30 15:57:16 -07:00
21263fb2ed
Lexer doesn't have a named_char module anymore
2017-04-30 15:57:02 -07:00
a88bf20a23
Derive Eq on lexer::number::Sign
2017-04-30 15:56:49 -07:00
28ba98582a
More compact formatting
2017-04-30 15:56:32 -07:00
9183e05891
Add FromChar trait to lexer::char
2017-04-30 15:56:21 -07:00
3ffb694ce1
Move Exact to types::number
2017-04-30 15:55:50 -07:00
fd5df91e27
[lexer,types] Move named_char to types as types/char/names.rs
...
Move char.rs to be char/mod.rs
2017-04-23 10:41:47 -07:00
5cb66d932f
[lexer] Clean up named_char module
2017-04-23 10:02:39 -07:00
a5f88c221d
Use Objects for Token values
2017-04-22 15:21:13 -07:00
9f1500c1b6
Improve logging ever so slightly
2017-04-15 18:12:24 -07:00
ddf13f9bae
Make Lex less mutable
2017-04-15 18:12:24 -07:00
f9d9a08455
Add a failing test that the lexer finds rational numbers
2017-04-15 10:35:13 -07:00
4f11be1316
Remove a println!
2017-04-15 10:17:50 -07:00
50baf2c118
Make all the lexer tests pass again
...
Try to make exact numbers if possible...
2017-04-15 09:46:21 -07:00
