sibil

eryn/sibil

Fork 0

Commit graph

59f9b5dd97 Dot state Eryn Wells 2016-12-26 11:51:03 -07:00
7b4d3c62b9 Check single character identifiers: +, - Eryn Wells 2016-12-26 11:37:45 -07:00
d63494fc32 Check that forward progress is always made by lexer states Eryn Wells 2016-12-26 09:15:53 -07:00
b0b9a38932 Straighten out test failures Eryn Wells 2016-12-26 09:15:43 -07:00
0a9da3b7e8 Assert invalid characters in states Eryn Wells 2016-12-26 08:52:15 -07:00
dfc528e8e7 Find left vector parens Eryn Wells 2016-12-25 20:59:21 -07:00
c42d69bd47 Lex dots: . Eryn Wells 2016-12-25 20:54:47 -07:00
c34064f51a TODO about using Results Eryn Wells 2016-12-25 20:49:46 -07:00
3d369daa66 Check single tokens in the lexer Eryn Wells 2016-12-25 20:49:15 -07:00
5a28b06434 Move lexer tests to a #[cfg(test)] mod tests { } modules Eryn Wells 2016-12-25 19:23:01 -07:00
81a84a796a Lex basic strings (no escapes yet) Eryn Wells 2016-12-25 15:03:18 -07:00
ea4758e442 Clean up end of next() Eryn Wells 2016-12-25 14:41:06 -07:00
953870fae0 Check for EOF ("\0") when reading comments Eryn Wells 2016-12-25 14:34:11 -07:00
b2f7afef07 Lexer::new takes a &str instead of a String Eryn Wells 2016-12-25 14:33:58 -07:00
a74228d11a Add char::is_eof() Eryn Wells 2016-12-25 14:33:26 -07:00
7f19c78127 Make some unit tests for the lexer Eryn Wells 2016-12-25 14:20:25 -07:00
7e784ffce1 Make the Lex at the end, make tokens in state methods Eryn Wells 2016-12-25 14:20:16 -07:00
0d89cfefc1 Add Comment token Eryn Wells 2016-12-25 13:50:34 -07:00
c925939faf Break apart is_boolean test into true/false variants Eryn Wells 2016-12-25 12:24:31 -07:00
acbde03786 Token is an enum with values; Lex contains a token and several useful bits of info Eryn Wells 2016-12-25 12:24:04 -07:00
6b8ba6b6e8 use #[derive(Debug)] on Token, Kind, State... SO MUCH EASIER Eryn Wells 2016-12-24 14:03:37 -07:00
c84251b56f Display for State enum Eryn Wells 2016-12-24 10:29:24 -07:00
43054b3f49 Lex hashes and booleans Eryn Wells 2016-12-24 10:29:10 -07:00
e9b0eab38a Sort out advancing pointers for whitespace Eryn Wells 2016-12-24 09:59:35 -07:00
8140e3c5e6 Pass over whitespace and count lines Eryn Wells 2016-12-24 09:17:08 -07:00
f7c28c6732 Add single character identifiers Eryn Wells 2016-12-24 09:07:38 -07:00
fd3f6f9f29 Remove characters module declaration Eryn Wells 2016-12-24 09:07:20 -07:00
f7cedb3536 Doc clean up Eryn Wells 2016-12-24 09:05:10 -07:00
d4dee92904 Move all the lexer stuff to a module directory Eryn Wells 2016-12-24 08:57:37 -07:00
d333819dee Break token stuff into a separate source file Eryn Wells 2016-12-24 08:47:02 -07:00
8d9cfd86f8 Move lexer.rs to lexer/mod.rs Eryn Wells 2016-12-24 08:41:18 -07:00
f1d15d5cef Documentation and some clean up Eryn Wells 2016-12-23 17:53:28 -07:00
089425ab0f Remove the demo stuff Eryn Wells 2016-12-23 17:48:19 -07:00
e6ca40122e Print out tokens as they come Eryn Wells 2016-12-23 17:46:36 -07:00
b1051f5f74 Do actual (quite simplified) lexing! Eryn Wells 2016-12-23 17:46:28 -07:00
9561e1bf91 Add Display trait to Token and Kind Eryn Wells 2016-12-23 17:45:37 -07:00
79080ff62b CharAt trait, applied to str Eryn Wells 2016-12-23 17:44:44 -07:00
fcb7a79721 Some unit tests for the RelativeIndexable methods Eryn Wells 2016-12-23 17:44:28 -07:00
4bd1d2c9c2 Redo how index_before and index_after work Eryn Wells 2016-12-23 17:43:04 -07:00
fa2ff9e8ce chars are Lexable Eryn Wells 2016-12-23 17:38:33 -07:00
e13b2c24de Laying some groundwork for actual lexing... Eryn Wells 2016-12-22 09:25:31 -08:00
2c9b024ac5 index_before and index_after return Option<usize> instead of just usize Eryn Wells 2016-12-22 09:25:10 -08:00
93353f0f9e Documentation for the set functions in characters.rs Eryn Wells 2016-12-22 09:24:53 -08:00
fbf1d07e27 Iterate lexer Eryn Wells 2016-12-20 17:52:46 -08:00
7de54a9835 Make public lexer bits public Eryn Wells 2016-12-20 17:52:29 -08:00
837403949e Fix off by one Eryn Wells 2016-12-20 17:52:00 -08:00
08bbaeb7cf Make ascii_* sets private to characters Eryn Wells 2016-12-20 17:51:43 -08:00
7c43e93470 Rustify my syntax Eryn Wells 2016-12-20 17:51:13 -08:00
f02240da87 Add characters module Eryn Wells 2016-12-20 17:38:44 -08:00
ac4cff9a51 Add a bunch of character stuff to the lexer module Eryn Wells 2016-12-19 22:23:27 -08:00
9e51659a30 Ignore Rust build artifacts Eryn Wells 2016-12-19 14:02:19 -08:00
a5220fdbaf Add initial sibil Rust project Eryn Wells 2016-12-19 14:01:55 -08:00
ed27494783 Try to lex identifiers and parens swift Eryn Wells 2016-12-19 09:46:58 -08:00
59e55a39e4 Straighten out Characters and CharacterSets Eryn Wells 2016-12-19 08:57:59 -08:00
35a5cf1dc8 Fiddling with state, I dunno Eryn Wells 2016-12-17 10:04:35 -08:00
aed0d3e43d Simple test harness Eryn Wells 2016-12-16 10:10:13 -08:00
a0eb34f9a2 Lex simple parentheses Eryn Wells 2016-12-16 10:10:05 -08:00
3840f9ee02 Add very rudimentary Lexer: it matches parens! Eryn Wells 2016-12-16 09:44:05 -08:00
06eb09ffc6 Add an Xcode project Eryn Wells 2016-12-16 08:52:16 -08:00
0416c6ac7d Initial commit Eryn Wells 2016-12-19 13:51:50 -08:00