Fix size_hint underflow on empty-string sentence iterators (#181) USentenceBounds::size_hint subtracted 1 from the inner iterator's lower bound, which is 0 for an empty string, so size_hint() on any of the three public sentence iterators panicked with 'attempt to subtract with overflow' in debug builds and returned a lower bound of usize::MAX in release builds. The cmp::max(0, ...) wrappers around the expression are no-ops on usize and hid the underflow. Both bounds now use saturating_sub(1). The regression test drives size_hint on every public iterator of the crate over a corpus led by the empty string and asserts lower <= yielded <= upper in both profiles.
Iterators which split strings on Grapheme Cluster, Word, or Sentence boundaries, according to the Unicode Standard Annex #29 rules.
use unicode_segmentation::UnicodeSegmentation; fn main() { let s = "a̐éö̲\r\n"; let g = s.graphemes(true).collect::<Vec<&str>>(); let b: &[_] = &["a̐", "é", "ö̲", "\r\n"]; assert_eq!(g, b); let s = "The quick (\"brown\") fox can't jump 32.3 feet, right?"; let w = s.unicode_words().collect::<Vec<&str>>(); let b: &[_] = &["The", "quick", "brown", "fox", "can't", "jump", "32.3", "feet", "right"]; assert_eq!(w, b); let s = "The quick (\"brown\") fox"; let w = s.split_word_bounds().collect::<Vec<&str>>(); let b: &[_] = &["The", " ", "quick", " ", "(", "\"", "brown", "\"", ")", " ", "fox"]; assert_eq!(w, b); }
unicode-segmentation does not depend on libstd, so it can be used in crates with the #![no_std] attribute.
You can use this package in your project by adding the following to your Cargo.toml:
[dependencies] unicode-segmentation = "1"
Yanked due to accidental breakage and MSRV mistag.
#[inline] opportunities, resulting in 15-40% performance improvement.GraphemeCursor API allows random access and bidirectional iteration.as_str methods to the iterator types.