Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21657
Actually the issue happened because I removed the mapping:
https://github.com/apache/spark/commit/64fad0b519cf35b8c0a0dec18dd3df9488a5ed25#diff-d19881aceddcaa5c60620fdcda99b4c4L79
I would propose to revert it back, and remove all those "expensive"
(comparing to look up `tokenIndexArr`) `map`s and the projection:
https://github.com/apache/spark/pull/21657/files#diff-a549ac2e19ee7486911e2e6403444d9dR156
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
