[ https://issues.apache.org/jira/browse/TIKA-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luís Filipe Nassif updated TIKA-3237: ------------------------------------- Description: There is a huge overhead in ForkParser ContentHandlerProxy and ContentHandlerResource read/write char[]/string methods. A simple change to not loop reading/writing each char of strings can result in speed ups of several orders of magnitude. Tests with ~10k small RFC822 files (22MB), 24 parsing threads, have shown parse times below: ForkParser off: 4s ForkParser on: 30min ForkParser patched: 17s Also, there is a bug when running at least with Oracle JDK8 that, if a String greater than 65535 chars is written, a UTFDataFormatException is thrown from DataOutputStream.writeUTF(String) method. was: There is a huge overhead in ForkParser ContentHandlerProxy and ContentHandlerResource read/write char[]/string methods. A simple change to not loop reading/writing each char of strings can result in speed ups of several orders of magnitude. Tests with ~10k small RFC822 files (22MB), 24 parsing threads, have shown parse times below: ForkParser off: 4s ForkParser on: 30min ForkParser patched: 17s Also, there is a bug when running at least with Oracle JDK8 that, if a String greater than 65535 is written, a UTFDataFormatException is thrown from DataOutputStream.writeUTF(String) method. > Great optimization in ForkParser > -------------------------------- > > Key: TIKA-3237 > URL: https://issues.apache.org/jira/browse/TIKA-3237 > Project: Tika > Issue Type: Improvement > Components: core > Affects Versions: 1.24.1 > Environment: Windows 10, Oracle JDK 1.8.0_261 x64, 24 cores > Reporter: Luís Filipe Nassif > Assignee: Luís Filipe Nassif > Priority: Major > > There is a huge overhead in ForkParser ContentHandlerProxy and > ContentHandlerResource read/write char[]/string methods. A simple change to > not loop reading/writing each char of strings can result in speed ups of > several orders of magnitude. > > Tests with ~10k small RFC822 files (22MB), 24 parsing threads, have shown > parse times below: > ForkParser off: 4s > ForkParser on: 30min > ForkParser patched: 17s > > Also, there is a bug when running at least with Oracle JDK8 that, if a String > greater than 65535 chars is written, a UTFDataFormatException is thrown from > DataOutputStream.writeUTF(String) method. -- This message was sent by Atlassian Jira (v8.3.4#803005)