Dear Hogan,
thanks for the hint.
I wrote my own Actor now that reads the whole file without requiring
caching.
Greetings,
Stefan
Am 2012-05-22 16:11, schrieb Hogan, D. (GE Energy):
The org.geon.BinaryFileReader actor has a limit of 20,000 bytes per
iteration. If you use that actor, you need to either process in up to
20,000 byte chunks or cache the results until the endOfFile port is
true.
Not all actors have this behavior. For instance,
ptolemy.actor.lib.io.FileReader and
org.resurgence.actor.SimpleFileReader read the entire file as a
character stream and output a single string.
-----Original Message-----
From: kepler-users-boun...@kepler-project.org
[mailto:kepler-users-boun...@kepler-project.org] On Behalf Of Stefan
Proell
Sent: Tuesday, May 22, 2012 10:05 AM
To: kepler-users@kepler-project.org
Subject: Re: [kepler-users] Size Limitation of ArrayTokens?
Hi!
There seems to be a problem with the buffer in the BinaryFileReader
Class, which is limited to 20000 bytes, as you can see in the
corresponding source file (line 277).
/** The current bytes read. */
protected byte[] bytesRead = new byte[20000];
It only is reading this first 20000 bytes and does not proceed after
this. I will have a closer look at ...
Stefan
Am 2012-05-22 15:12, schrieb Stefan Proell:
Dear Matt,
thanks for your example. It works fine, but does not solve my problem.
I use a Binary File Reader to read a file from my local disk. The
Binary File Reader reads a local file path or URL and outputs an array
of bytes. No matter what file I want to read, the array size is
exactly 20 000 (bytes). I attached a small sample workflow to
demonstrate this. It simply reads a file (in my testcase this file is
about 40 MB) and displays the length of the resulting Byte Array.
Do you have any idea why this behaviour occurs and can your reproduce
it?
Kind regards,
Stefan
Am 2012-05-21 19:05, schrieb Matt Jones:
I also took a look, and tried a larger array -- the attached workflow
processes 30K integers through an array and back out again, and works
fine. So there isn't an inherent limit at 20K elements. So I'm not
quite sure what's happening with yours, but maybe this will help to
debug.
Matt
On Mon, May 21, 2012 at 8:32 AM, Christopher Brooks
<c...@eecs.berkeley.edu<mailto:c...@eecs.berkeley.edu>> wrote:
HI Stefan,
A quick search of the web shows no apparent limits.
A more complete test case would help.
The way I would approach this is by splitting up the problem into
first just reading the data and being sure that worked and then
adding more like encoding.
_Christopher
On 5/21/12 9:20 AM, Stefan Proell wrote:
Dear Mailing list,
I wrote my own Actor which I need in order to encode binary
files via Base64 and pass the resulting encoded String to a
REST Service. I feed the binary file with a
BinaryFileReader-Actor to my encoding Actor and cast the
Token
to an Array, as I found no other solution to receive the
bytes
from the file. I then use a standard (and also deprecated)
method for encoding the file first to Base64 and then send it
through some URL-safe encoding. The method looks like this:
@Override
public void fire() throws IllegalActionException {
super.fire();
// Read file from InputPort and convert to a
ByteArray
ArrayToken inputToken = (ArrayToken) inputPort.get(0);
byte[] inputBytes =
ArrayToken.arrayTokenToUnsignedByteArray(inputToken);
// Encoding
String encode = new
String(Base64.encodeBase64(inputBytes));
String encodedString =
java.net.URLEncoder.encode(encode);
output.send(0, new StringToken(encodedString));
}
My problem is that the encoded file is truncated by the actor
and hence not usable for further processing. There seems to
be
a limit of 20 000 array elements (bytes), which is not
sufficient for my purpose. Does anyone have an idea why the
file is chopped off after 20k letters?
Thanks in advance,
Stefan
_______________________________________________
Kepler-users mailing list
Kepler-users@kepler-project.org
<mailto:Kepler-users@kepler-project.org>
http://lists.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
-- Christopher Brooks, PMP University
of California
CHESS Executive Director US Mail: 337 Cory
Hall
Programmer/Analyst CHESS/Ptolemy/Trust Berkeley, CA
94720-1774
ph: 510.643.9841<tel:510.643.9841>
(Office: 545Q Cory)
home: (F-Tu) 707.665.0131<tel:707.665.0131> cell: 707.332.0670
<tel:707.332.0670>
_______________________________________________
Kepler-users mailing list
Kepler-users@kepler-project.org
<mailto:Kepler-users@kepler-project.org>
http://lists.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
_______________________________________________
Kepler-users mailing list
Kepler-users@kepler-project.org
http://lists.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
_______________________________________________
Kepler-users mailing list
Kepler-users@kepler-project.org
http://lists.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
_______________________________________________
Kepler-users mailing list
Kepler-users@kepler-project.org
http://lists.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
