tokenize

Tokenises an Algosim expression.

Syntax

• tokenize(s)

  • s is a string

Description

If s is a string containing an Algosim expression, then tokenize(s) returns a list of all tokens found in s, each token being summarised in a few words.

Examples

tokenize("n^2/(√3 + 1)")

Identifier "n"
Infix operator ^
Integer literal 2
Infix operator /
Bracket ( – partner at idx 9
Prefix operator √
Integer literal 3
Infix operator +
Integer literal 1
Bracket ) – partner at idx 4

See also

• parse