OK - we know that content1.pdf is choking b/c of the embedded
images. To fix that, the PdfContentParser class would have to be
enhanced to properly handle those. This is outside of my scope, but
maybe someone else on the list is up for trying to add it.
There is a lot of content in tic_doug2.pdf, which makes it hard to
develop a decent use case - as things progress, it may be
appropriate to generate a sample PDF that contains just one font and
some pretty simple content (maybe two or three lines of text with 4
or 5 words each). But given the PDF we have now, the thing that is
failing (at least for me) is loading of one of the fonts (F1).
Here is the info from the fonts dictionary:
Subdictionary /F1 = (/DescendantFonts=[4 0 R], /BaseFont=/GothicBBB-
Medium-Identity-H, /Type=/Font, /Encoding=/Identity-H, /Subtype=/
Type0)
DocumentFont recognizes this as a Type0 font, and attempts to parse
it. However, it chokes b/c the ToUnicode map is not specified for
the font. As we know, this is expected, because it should be
possible to obtain a CMap from elsewhere in the system, or intuit
the CMap based on information in the font (is that right??). I have
a patch (below) that attempts to work around this requirement. It's
not pretty, but it should tell us if we are on the right track or not.
So there are two pieces that have to be in place for this to work:
1. Ability to find the appropriate CMap resource
2. Ability to load and use the CMap
I'll take each in turn:
Finding the appropriate CMap resource
I'm not at all experienced with font dictionaries. But my initial
thinking is that the F1 font listed above probably does not require
an external CMap - the encoding specified is Identify-H, so
shouldn't that be the CMap we use? And isn't that CMap basically
just a one-to-one mapping of characters to bytes? Or are CMaps
intended as a supplimentary mapping technology, so is it more
appropriate to say there is no CMap, so we shouldn't be trying to
fill font metrics at all?
I've put together a patch for DocumentFont that just assumes an
identity mapping when there is no ToUnicode entry provided (if it
would be better to handle this sort of thing in an SVN branch,
please let me know. This patch removes the null pointer exception,
and attempts to build an appropriate metrics map. Note that this
implementation is horrendously inneficient ( we wind up creating a
metrics map that has every potential character in it) - but it
should at least prove out the concept.
If this is indeed what needs to be done in cases where an Identity-H
CMap is appropriate, then we can optimize... Be sure to read the
section that appears after the following patch (outlining changes to
DocumentFont to have it work with CMaps directly) - this is the
beginning of the change required to handle the non-tounicode
situation efficiently (without storing a map essentially saying 1=1,
2=2, 3=3, ... 0xffff = 0xffff).
With this patch in place, my text content parser does not fail - it
produces text strings. I do not have the font in question on my
system, so I can't tell if the actual extracted text is correct or
not - perhaps Michael can apply the patch and run the tic_doug2.pdf
file through PdfContentReaderTool and see what it gives him?
### Eclipse Workspace Patch 1.0
#P iText Trunk
Index: src/core/com/lowagie/text/pdf/DocumentFont.java
===================================================================
--- src/core/com/lowagie/text/pdf/DocumentFont.java (revision 3613)
+++ src/core/com/lowagie/text/pdf/DocumentFont.java (working copy)
@@ -155,7 +155,8 @@
private void processType0(PdfDictionary font) {
try {
- byte[] touni =
PdfReader
.getStreamBytes
((PRStream
)PdfReader.getPdfObjectRelease(font.get(PdfName.TOUNICODE)));
+ PdfObject toUnicodeReference =
font.get(PdfName.TOUNICODE);
+ byte[] touni = toUnicodeReference == null ? null :
PdfReader
.getStreamBytes
((PRStream )PdfReader.getPdfObjectRelease(toUnicodeReference));
PdfArray df =
(PdfArray
)PdfReader.getPdfObjectRelease(font.get(PdfName.DESCENDANTFONTS));
PdfDictionary cidft =
(PdfDictionary
)PdfReader.getPdfObjectRelease((PdfObject)df.getArrayList().get(0));
PdfNumber dwo =
(PdfNumber)PdfReader.getPdfObjectRelease(cidft.get(PdfName.DW));
@@ -204,6 +205,20 @@
}
private void fillMetrics(byte[] touni, IntHashtable widths, int
dw) {
+ if (touni == null){ // just assume a one-to-one mapping
+ // this is hideously inefficient - much better to use a
CMap object
+ ;
+ for(int i = 0; i <= 0xffff; i++){
+ int unic = i;
+ int w = dw;
+ if (widths.containsKey(unic))
+ w = widths.get(i);
+ metrics.put(new Integer(unic), new int[]{unic, w});
+ }
+
+ & nbsp; return;
+ }
+
try {
PdfContentParser ps = new PdfContentParser(new
PRTokeniser(touni));
PdfObject ob = null;
Loading and using the CMap
I think that the correct way to solve this is to make some changes
to DocumentFont.fillMetrics so that it works with CMap objects
instead of the unicode byte array. The CMap object would be
obtained (or at least attempted) during construction of DocumentFont
(regardless of the type of font!).
This will actually drastically simplify fillMetrics (this kind of
parsing probably doesn't belong inside a font object anyway), and it
will make the CMap available via the DocumentFont directly. If we
do this, then the need for CMapAwareDocumentFont is removed
entirely, and we can remove that class.
To keep things efficient, I could use some help with the change to
fillMetrics, specifically how to fill the 'metrics' map instance
variable given a CMap object - or does CMap remove the need for the
metrics map entirely? Maybe the map should just be a 'widths' map?
If it removes the need entirely, then I think adjusting DocumentFont
to just use CMap makes the DocumentFont source much more readable.
So, I think I see a way forward here - but I'm going to need some
help from the iText maintainers. Here's what I see as how to proceed:
1. add a loadCMap() method to DocumentFont (the results of the
'finding the appropriate CMap resource' discussion above will be
used for this)
2. call loadCMap() from DocumentFont(PRIndirectReference refFont)
3. adjust both getWidth() methods so they use CMap instead of metrics
4. adjust both convertToBytes() methods so they use CMap instead of
metrics
5. adjust charExists() so it uses CMap instead of metrics
6. add getCMap() to DocumentFont
7. move CMapAwareDocumentFont.encode() to DocumentFont (if this is
undesirable, we could continue to have this method in a sub-class -
it just seems more intuitive to have it in DocumentFont)
once that is in place, I can:
8. adjust PdfContentStreamProcessor so it uses DocumentFont directly
9. remove CMapAwareDocumentFont from the code base (depends on step
7 above)
What do you all think?
- K
----------------------- Original Message -----------------------
From: "Hoppe, Michael" <michael.ho...@fiz-karlsruhe.de>
To: "Post all your questions about iText here" <itext-questions@lists.sourceforge.net
>
Cc:
Date: Wed, 17 Dec 2008 17:12:58 +0100
Subject: Re: [iText-questions] extracting text from pdfs with
japanese data
Hi all,
Attached see the Pdfs i had the problems with (I send them once
before)
content1.pdf gives : java.io.IOException: '>' not expected at file
pointer 39040
tic_dogu2.pdf gives java.lang.NullPointerException because font is
not embedded in pdf
text from content1.pdf can get extracted with the adobe viewer bean
(another open source library that we don’t want to use for our
project for various reasons) so I don’t think there is something
wrong with the file itself.
Greetings
Michael
Dr. Michael Hoppe
ePublishing & eScience
Development & Applied Research
Phone +49 7247 808-251
Fax +49 7247 808-133
michael.ho...@fiz-karlsruhe.de
FIZ Karlsruhe
Hermann-von-Helmholtz-Platz 1
76344 Eggenstein-Leopoldshafen, Germany
www.fiz-karlsruhe.de
Von: Kevin Day [mailto:ke...@trumpetinc.com]
Gesendet: Mittwoch, 17. Dezember 2008 15:31
An: IText Questions
Betreff: Re: [iText-questions] extracting text from pdfs with
japanese data
CMapAwareDocumentFont has this parsing via the CMap class - this
encapsulates the parsing behind an object, and makes it a lot easier
to deal with.
I think that the biggest thing here is actually finding the
appropriate CMap data byte stream (either from embedded data in the
PDF, or from the file system) - right now, locating the CMap
information is a weak point in the content parser.
If the cmap data is included in a jar on the classpath, then the
CMap could absolutely be read from the jar.
Can the OP please send a PDF that demonstrates the issue? I'll take
a look at the font information and see how tough it would be to add
this type of lookup if TOUNICODE isn't available.
- K
----------------------- Original Message -----------------------
From: "Paulo Soares" <psoa...@consiste.pt>
To: "Post all your questions about iText here" <itext-questions@lists.sourceforge.net
>
Cc:
Date: Tue, 16 Dec 2008 09:55:36 -0000
Subject: Re: [iText-questions] extracting text from pdfs with
japanese data
There's code in PdfEncodings to parse and convert to/from Unicode
the cmaps.
The font contains the cmap name.
Paulo
----- Original Message -----
From: "1T3XT info" <i...@1t3xt.info>
To: "Post all your questions about iText here"
<itext-questions@lists.sourceforge.net>
Sent: Tuesday, December 16, 2008 9:19 AM
Subject: Re: [iText-questions] extracting text from pdfs with
japanese data
Hoppe, Michael wrote:
> The CMap-files are included in the iTextAsianCmaps.jar. So
couldn’t they
> be read from that jar in case there is no font information in the
pdf?
I'm just thinking out loud here, I didn't dive into the problem yet,
but: do you think it's possible for iText to find which CMap-file is
t o
be inspected based on the font information availa ble in the PDF?
As Kevin already said: this part of iText is pretty new. We're all
excited about it, but for the moment it's all highly experimental.
--
This answer is provided by 1T3XT BVBA
http://www.1t3xt.com/ - http://www.1t3xt.info
------------------------------------------------------------------------------
SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas,
Nevada.
The future of the web can't happen without you. Join us at MIX09 to
help
pave the way to the Next Web now. Learn more and register at
http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
_______________________________________________
iText-questions mailing list
iText-questions@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/itext-questions
Buy the iText book: http://www.1t3xt.com/docs/book.php
-------------------------------------------------------
Fachinformationszentrum Karlsruhe, Gesellschaft für wissenschaftlich-
technische Information mbH.
Sitz der Gesellschaft: Eggenstein-Leopoldshafen, Amtsgericht
Mannheim HRB 101892.
Geschäftsführerin: Sabine Brünger-Weilandt.
Vorsitzender des Aufsichtsrats: MinR Hermann Riehl.
------------------------------------------------------------------------------
SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas,
Nevada.
The future of the web can't happen without you. Join us at MIX09 to
help
pave the way to the Next Web now. Learn more and register at
http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
_______________________________________________
iText-questions mailing list
iText-questions@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/itext-questions
Buy the iText book: http://www.1t3xt.com/docs/book.php
------------------------------------------------------------------------------
SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas,
Nevada.
The future of the web can't happen without you. Join us at MIX09 to
help
pave the way to the Next Web now. Learn more and register at
http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/_______________________________________________
iText-questions mailing list
iText-questions@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/itext-questions
Buy the iText book: http://www.1t3xt.com/docs/book.php
