> On 二月 12, 2016, 9:35 a.m., Ben Mahler wrote: > > 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp, lines 136-171 > > <https://reviews.apache.org/r/43469/diff/2/?file=1240148#file1240148line136> > > > > How about the following to simplify the logic and make it consistent > > with strings::split? Note that I noticed a bug in strings::split for n=1 > > and filed [MESOS-4656](https://issues.apache.org/jira/browse/MESOS-4656). > > > > ``` > > inline std::vector<std::string> tokenize( > > const std::string& s, > > const std::string& delims, > > const Option<size_t>& maxTokens = None()) > > { > > size_t offset = 0; > > std::vector<std::string> tokens; > > > > while (maxTokens.isNone() || maxTokens.get() > 0) { > > size_t nonDelim = s.find_first_not_of(delims, offset); > > > > if (nonDelim == std::string::npos) { > > break; // Nothing left > > } > > > > size_t delim = s.find_first_of(delims, nonDelim); > > > > // Finish tokenizing if this is the last token, > > // or we've found enough tokens. > > if (delim == std::string::npos || > > (maxTokens.isSome() && tokens.size() == maxTokens.get() - 1)) { > > tokens.push_back(s.substr(nonDelim)); > > break; > > } > > > > tokens.push_back(s.substr(nonDelim, delim - nonDelim)); > > offset = delim; > > } > > > > return tokens; > > } > > ```
Thanks Ben, this looks simpler and cleaner! I also did some test for the updated patch and it works well with the unit test cases. - Guangya ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43469/#review118988 ----------------------------------------------------------- On 二月 11, 2016, 8:32 a.m., Guangya Liu wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/43469/ > ----------------------------------------------------------- > > (Updated 二月 11, 2016, 8:32 a.m.) > > > Review request for mesos and Ben Mahler. > > > Bugs: MESOS-3833 > https://issues.apache.org/jira/browse/MESOS-3833 > > > Repository: mesos > > > Description > ------- > > Added a parameter to specify the maximum number of tokens for tokenize. > > > Diffs > ----- > > 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp > c20e2ffe128059f9999e63ac2630a2c73e512fc6 > 3rdparty/libprocess/3rdparty/stout/tests/strings_tests.cpp > 8e7d3eeb22645c7600df1ee215a9de6a26166096 > > Diff: https://reviews.apache.org/r/43469/diff/ > > > Testing > ------- > > LiuGuangyas-MacBook-Pro:3rdparty gyliu$ ./stout-tests > --gtest_filter="StringsTest.Tokenize*" --verbose > Note: Google Test filter = StringsTest.Tokenize* > [==========] Running 15 tests from 1 test case. > [----------] Global test environment set-up. > [----------] 15 tests from StringsTest > [ RUN ] StringsTest.Tokenize > [ OK ] StringsTest.Tokenize (0 ms) > [ RUN ] StringsTest.TokenizeStringWithDelimsAtStart > [ OK ] StringsTest.TokenizeStringWithDelimsAtStart (0 ms) > [ RUN ] StringsTest.TokenizeStringWithDelimsAtEnd > [ OK ] StringsTest.TokenizeStringWithDelimsAtEnd (0 ms) > [ RUN ] StringsTest.TokenizeStringWithDelimsAtStartAndEnd > [ OK ] StringsTest.TokenizeStringWithDelimsAtStartAndEnd (0 ms) > [ RUN ] StringsTest.TokenizeWithMultipleDelims > [ OK ] StringsTest.TokenizeWithMultipleDelims (0 ms) > [ RUN ] StringsTest.TokenizeEmptyString > [ OK ] StringsTest.TokenizeEmptyString (0 ms) > [ RUN ] StringsTest.TokenizeDelimOnlyString > [ OK ] StringsTest.TokenizeDelimOnlyString (0 ms) > [ RUN ] StringsTest.TokenizeNullByteDelim > [ OK ] StringsTest.TokenizeNullByteDelim (0 ms) > [ RUN ] StringsTest.TokenizeNZero > [ OK ] StringsTest.TokenizeNZero (0 ms) > [ RUN ] StringsTest.TokenizeNDelimOnlyString > [ OK ] StringsTest.TokenizeNDelimOnlyString (0 ms) > [ RUN ] StringsTest.TokenizeN > [ OK ] StringsTest.TokenizeN (0 ms) > [ RUN ] StringsTest.TokenizeNStringWithDelimsAtStart > [ OK ] StringsTest.TokenizeNStringWithDelimsAtStart (0 ms) > [ RUN ] StringsTest.TokenizeNStringWithDelimsAtEnd > [ OK ] StringsTest.TokenizeNStringWithDelimsAtEnd (0 ms) > [ RUN ] StringsTest.TokenizeNStringWithDelimsAtStartAndEnd > [ OK ] StringsTest.TokenizeNStringWithDelimsAtStartAndEnd (0 ms) > [ RUN ] StringsTest.TokenizeNWithMultipleDelims > [ OK ] StringsTest.TokenizeNWithMultipleDelims (0 ms) > [----------] 15 tests from StringsTest (0 ms total) > > [----------] Global test environment tear-down > [==========] 15 tests from 1 test case ran. (0 ms total) > [ PASSED ] 15 tests. > > > Thanks, > > Guangya Liu > >
