Re: Confused about one read method in TsFileInput interface
yes, The method does not change the position of the FileChannel Reader. Add it in the javadoc is a good idea. --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Yuan Tian 于2019年9月9日周一 下午9:33写道： > Hi, > > While implementing one read method(shown in the following) in TsFileInput > interface, I don’t know whether to modify the position of TsFileInput. > > /** > * read an array of byte from the Input. > * > * @param b -array of byte > * @param off -offset of the Input > * @param len -length > */ > int read(byte[] b, int off, int len) throws IOException; > > > I think this method should not modify this TsFileInput and it should be > specified in the comments. > > > Best, > -- > Yuan Tian > School of Software, Tsinghua University > > 田原 > 清华大学 软件学院 > > > > >
Re: Confused about one read method in TsFileInput interface
Hi,
- `TsFileInput.read(ByteBuffer dst, long position)` does not modify this
TsFileInput's position.
It works just as the annotation depicts.
If you do a simple experiment using the `class DefaultTsFileInput implements
TsFileInput` to
1) position(), 2) read(ByteBuffer dst, long position), 3) position(),
you will find that the two positions before and after read are the same.
- `class HDFSInput implements TsFileInput` in the tsfile-spark module
I encountered this similar question when implementing tsfile-spark-connector
before.
As you can see in the `class HDFSInput implements TsFileInput` in the
tsfile-spark module,
I use a plain method to realize the `read(ByteBuffer dst, long position)`
method.
Sincerely,
Lei Rui
On 9/10/2019 00:21，Yuan Tian wrote：
Hi, Boris
Yes, like what I said, the method without position signature acts like the
inputStream that you can’t manually change the position where you want to read.
Instead, you can only read by order. The position will automatically move
forward while you read data from the stream.
Best,
--
Yuan Tian
School of Software, Tsinghua University
田原
清华大学 软件学院
在 2019年9月10日，上午12:09，Boris Zhu 写道：
I check-in java doc and it says the file position is updated with the
number of bytes actually read. You can see below
* Reads a sequence of bytes from this channel into the given buffer.
*
* Bytes are read starting at this channel's current file position, and
* then the file position is updated with the number of bytes actually
* read. Otherwise this method behaves exactly as specified in the {@link
* ReadableByteChannel} interface.
*/
public abstract int read(ByteBuffer dst) throws IOException;
you can also see it in
https://docs.oracle.com/javase/8/docs/api/java/nio/channels/FileChannel.html#read-java.nio.ByteBuffer-
best,
Boris zhu
On Tue, Sep 10, 2019 at 12:00 AM Yuan Tian wrote:
Hi, Boris
If the position is not specified, I think the position should be modified.
There are two methods in TsFileInput interface that don’t have position
parameter in their function signature.
1.
int read(ByteBuffer dst) throws IOException;
2.
int read() throws IOException;
Actually, in the corresponding methods in the FileChannel, they really
don’t modify the position which you can verify by reading the java doc of
that class. Perhaps, the reason for that is when you use the method without
position parameter, you just want to read from it by order like using a
input stream.
Best,
--
Yuan Tian
School of Software, Tsinghua University
田原
清华大学 软件学院
在 2019年9月9日，下午11:46，Boris Zhu 写道：
I agree with you that shouldn’t modify the position of TsFileInput when
having already specified the position. Sorry for the misunderstanding.
But
if not specified, should it be changed for hdfs or local?
Best,
Boris zhu
On Mon, Sep 9, 2019 at 11:35 PM Yuan Tian
wrote:
In fact, I am working on the hadoop-connector, so I should create a
HDFSInput that implements TsFileInput interface for reading tsfile from
HDFS.
And actually, in DefaultTsFileInput, this method is unsupported which
you
can see in the following.
@Override
public int read(byte[] b, int off, int len) throws IOException {
throw new UnsupportedOperationException();
}
Not all read methods in DefaultTsFileInput modify the position, such as
'int read(ByteBuffer dst, long position)’.
So it seems that if you specify the position where you want to start in
the method signature, the method shouldn’t modify the position of
TsFileInput which is consistent with FileChannel’s interface.
Best,
--
Yuan Tian
School of Software, Tsinghua University
田原
清华大学 软件学院
在 2019年9月9日，下午11:19，Boris Zhu 写道：
Hi, tian
DefaultTsFileInput is a class to implement TsFileInput
interface. DefaultTsFileInput's method read actually change the
position
of
itself because it uses java.io.FileChannel, and we can use a "position"
method to change its position back. But I'm also confusing that why
can't
change the position of TsFileInput or modify this TsFileInput.
Best,
Boris zhu
On Mon, Sep 9, 2019 at 10:55 PM Yuan Tian
wrote:
Hi,
While implementing one read method(shown in the following) in
TsFileInput
interface, I don’t know whether to modify the position of TsFileInput.
/**
* read an array of byte from the Input.
*
* @param b -array of byte
* @param off -offset of the Input
* @param len -length
*/
int read(byte[] b, int off, int len) throws IOException;
I think this method should not modify this TsFileInput and it should
be
specified in the comments.
Best,
--
Yuan Tian
School of Software, Tsinghua University
田原
清华大学 软件学院
Re: Confused about one read method in TsFileInput interface
Hi, Boris
Yes, like what I said, the method without position signature acts like the
inputStream that you can’t manually change the position where you want to read.
Instead, you can only read by order. The position will automatically move
forward while you read data from the stream.
Best,
--
Yuan Tian
School of Software, Tsinghua University
田原
清华大学 软件学院
> 在 2019年9月10日，上午12:09，Boris Zhu 写道：
>
> I check-in java doc and it says the file position is updated with the
> number of bytes actually read. You can see below
>
> * Reads a sequence of bytes from this channel into the given buffer.
> *
> * Bytes are read starting at this channel's current file position, and
> * then the file position is updated with the number of bytes actually
> * read. Otherwise this method behaves exactly as specified in the {@link
> * ReadableByteChannel} interface.
> */
> public abstract int read(ByteBuffer dst) throws IOException;
>
> you can also see it in
> https://docs.oracle.com/javase/8/docs/api/java/nio/channels/FileChannel.html#read-java.nio.ByteBuffer-
>
> best,
> Boris zhu
>
> On Tue, Sep 10, 2019 at 12:00 AM Yuan Tian wrote:
>
>> Hi, Boris
>>
>> If the position is not specified, I think the position should be modified.
>> There are two methods in TsFileInput interface that don’t have position
>> parameter in their function signature.
>>
>> 1.
>> int read(ByteBuffer dst) throws IOException;
>> 2.
>> int read() throws IOException;
>>
>> Actually, in the corresponding methods in the FileChannel, they really
>> don’t modify the position which you can verify by reading the java doc of
>> that class. Perhaps, the reason for that is when you use the method without
>> position parameter, you just want to read from it by order like using a
>> input stream.
>>
>>
>>
>> Best,
>> --
>> Yuan Tian
>> School of Software, Tsinghua University
>>
>> 田原
>> 清华大学 软件学院
>>
>>> 在 2019年9月9日，下午11:46，Boris Zhu 写道：
>>>
>>> I agree with you that shouldn’t modify the position of TsFileInput when
>>> having already specified the position. Sorry for the misunderstanding.
>> But
>>> if not specified, should it be changed for hdfs or local?
>>>
>>> Best,
>>> Boris zhu
>>>
>>> On Mon, Sep 9, 2019 at 11:35 PM Yuan Tian
>> wrote:
>>>
In fact, I am working on the hadoop-connector, so I should create a
HDFSInput that implements TsFileInput interface for reading tsfile from
HDFS.
And actually, in DefaultTsFileInput, this method is unsupported which
>> you
can see in the following.
@Override
public int read(byte[] b, int off, int len) throws IOException {
throw new UnsupportedOperationException();
}
Not all read methods in DefaultTsFileInput modify the position, such as
'int read(ByteBuffer dst, long position)’.
So it seems that if you specify the position where you want to start in
the method signature, the method shouldn’t modify the position of
TsFileInput which is consistent with FileChannel’s interface.
Best,
--
Yuan Tian
School of Software, Tsinghua University
田原
清华大学 软件学院
> 在 2019年9月9日，下午11:19，Boris Zhu 写道：
>
> Hi, tian
> DefaultTsFileInput is a class to implement TsFileInput
> interface. DefaultTsFileInput's method read actually change the
>> position
of
> itself because it uses java.io.FileChannel, and we can use a "position"
> method to change its position back. But I'm also confusing that why
>> can't
> change the position of TsFileInput or modify this TsFileInput.
>
> Best,
> Boris zhu
>
> On Mon, Sep 9, 2019 at 10:55 PM Yuan Tian
wrote:
>
>> Hi,
>>
>> While implementing one read method(shown in the following) in
TsFileInput
>> interface, I don’t know whether to modify the position of TsFileInput.
>>
>> /**
>> * read an array of byte from the Input.
>> *
>> * @param b -array of byte
>> * @param off -offset of the Input
>> * @param len -length
>> */
>> int read(byte[] b, int off, int len) throws IOException;
>>
>>
>> I think this method should not modify this TsFileInput and it should
>> be
>> specified in the comments.
>>
>> Best,
>> --
>> Yuan Tian
>> School of Software, Tsinghua University
>>
>> 田原
>> 清华大学 软件学院
>>
>>
>>
>>
Re: Confused about one read method in TsFileInput interface
I check-in java doc and it says the file position is updated with the
number of bytes actually read. You can see below
* Reads a sequence of bytes from this channel into the given buffer.
*
* Bytes are read starting at this channel's current file position, and
* then the file position is updated with the number of bytes actually
* read. Otherwise this method behaves exactly as specified in the {@link
* ReadableByteChannel} interface.
*/
public abstract int read(ByteBuffer dst) throws IOException;
you can also see it in
https://docs.oracle.com/javase/8/docs/api/java/nio/channels/FileChannel.html#read-java.nio.ByteBuffer-
best,
Boris zhu
On Tue, Sep 10, 2019 at 12:00 AM Yuan Tian wrote:
> Hi, Boris
>
> If the position is not specified, I think the position should be modified.
> There are two methods in TsFileInput interface that don’t have position
> parameter in their function signature.
>
> 1.
> int read(ByteBuffer dst) throws IOException;
> 2.
> int read() throws IOException;
>
> Actually, in the corresponding methods in the FileChannel, they really
> don’t modify the position which you can verify by reading the java doc of
> that class. Perhaps, the reason for that is when you use the method without
> position parameter, you just want to read from it by order like using a
> input stream.
>
>
>
> Best,
> --
> Yuan Tian
> School of Software, Tsinghua University
>
> 田原
> 清华大学 软件学院
>
> > 在 2019年9月9日，下午11:46，Boris Zhu 写道：
> >
> > I agree with you that shouldn’t modify the position of TsFileInput when
> > having already specified the position. Sorry for the misunderstanding.
> But
> > if not specified, should it be changed for hdfs or local?
> >
> > Best,
> > Boris zhu
> >
> > On Mon, Sep 9, 2019 at 11:35 PM Yuan Tian
> wrote:
> >
> >> In fact, I am working on the hadoop-connector, so I should create a
> >> HDFSInput that implements TsFileInput interface for reading tsfile from
> >> HDFS.
> >>
> >> And actually, in DefaultTsFileInput, this method is unsupported which
> you
> >> can see in the following.
> >>
> >> @Override
> >> public int read(byte[] b, int off, int len) throws IOException {
> >>throw new UnsupportedOperationException();
> >> }
> >>
> >> Not all read methods in DefaultTsFileInput modify the position, such as
> >> 'int read(ByteBuffer dst, long position)’.
> >>
> >> So it seems that if you specify the position where you want to start in
> >> the method signature, the method shouldn’t modify the position of
> >> TsFileInput which is consistent with FileChannel’s interface.
> >>
> >>
> >> Best,
> >> --
> >> Yuan Tian
> >> School of Software, Tsinghua University
> >>
> >> 田原
> >> 清华大学 软件学院
> >>
> >>> 在 2019年9月9日，下午11:19，Boris Zhu 写道：
> >>>
> >>> Hi, tian
> >>> DefaultTsFileInput is a class to implement TsFileInput
> >>> interface. DefaultTsFileInput's method read actually change the
> position
> >> of
> >>> itself because it uses java.io.FileChannel, and we can use a "position"
> >>> method to change its position back. But I'm also confusing that why
> can't
> >>> change the position of TsFileInput or modify this TsFileInput.
> >>>
> >>> Best,
> >>> Boris zhu
> >>>
> >>> On Mon, Sep 9, 2019 at 10:55 PM Yuan Tian
> >> wrote:
> >>>
> Hi,
>
> While implementing one read method(shown in the following) in
> >> TsFileInput
> interface, I don’t know whether to modify the position of TsFileInput.
>
> /**
> * read an array of byte from the Input.
> *
> * @param b -array of byte
> * @param off -offset of the Input
> * @param len -length
> */
> int read(byte[] b, int off, int len) throws IOException;
>
>
> I think this method should not modify this TsFileInput and it should
> be
> specified in the comments.
>
> Best,
> --
> Yuan Tian
> School of Software, Tsinghua University
>
> 田原
> 清华大学 软件学院
>
>
> >>
> >>
>
>
Re: Confused about one read method in TsFileInput interface
Hi, Boris
If the position is not specified, I think the position should be modified.
There are two methods in TsFileInput interface that don’t have position
parameter in their function signature.
1.
int read(ByteBuffer dst) throws IOException;
2.
int read() throws IOException;
Actually, in the corresponding methods in the FileChannel, they really don’t
modify the position which you can verify by reading the java doc of that class.
Perhaps, the reason for that is when you use the method without position
parameter, you just want to read from it by order like using a input stream.
Best,
--
Yuan Tian
School of Software, Tsinghua University
田原
清华大学 软件学院
> 在 2019年9月9日，下午11:46，Boris Zhu 写道：
>
> I agree with you that shouldn’t modify the position of TsFileInput when
> having already specified the position. Sorry for the misunderstanding. But
> if not specified, should it be changed for hdfs or local?
>
> Best,
> Boris zhu
>
> On Mon, Sep 9, 2019 at 11:35 PM Yuan Tian wrote:
>
>> In fact, I am working on the hadoop-connector, so I should create a
>> HDFSInput that implements TsFileInput interface for reading tsfile from
>> HDFS.
>>
>> And actually, in DefaultTsFileInput, this method is unsupported which you
>> can see in the following.
>>
>> @Override
>> public int read(byte[] b, int off, int len) throws IOException {
>>throw new UnsupportedOperationException();
>> }
>>
>> Not all read methods in DefaultTsFileInput modify the position, such as
>> 'int read(ByteBuffer dst, long position)’.
>>
>> So it seems that if you specify the position where you want to start in
>> the method signature, the method shouldn’t modify the position of
>> TsFileInput which is consistent with FileChannel’s interface.
>>
>>
>> Best,
>> --
>> Yuan Tian
>> School of Software, Tsinghua University
>>
>> 田原
>> 清华大学 软件学院
>>
>>> 在 2019年9月9日，下午11:19，Boris Zhu 写道：
>>>
>>> Hi, tian
>>> DefaultTsFileInput is a class to implement TsFileInput
>>> interface. DefaultTsFileInput's method read actually change the position
>> of
>>> itself because it uses java.io.FileChannel, and we can use a "position"
>>> method to change its position back. But I'm also confusing that why can't
>>> change the position of TsFileInput or modify this TsFileInput.
>>>
>>> Best,
>>> Boris zhu
>>>
>>> On Mon, Sep 9, 2019 at 10:55 PM Yuan Tian
>> wrote:
>>>
Hi,
While implementing one read method(shown in the following) in
>> TsFileInput
interface, I don’t know whether to modify the position of TsFileInput.
/**
* read an array of byte from the Input.
*
* @param b -array of byte
* @param off -offset of the Input
* @param len -length
*/
int read(byte[] b, int off, int len) throws IOException;
I think this method should not modify this TsFileInput and it should be
specified in the comments.
Best,
--
Yuan Tian
School of Software, Tsinghua University
田原
清华大学 软件学院
>>
>>
Re: Confused about one read method in TsFileInput interface
I agree with you that shouldn’t modify the position of TsFileInput when
having already specified the position. Sorry for the misunderstanding. But
if not specified, should it be changed for hdfs or local?
Best,
Boris zhu
On Mon, Sep 9, 2019 at 11:35 PM Yuan Tian wrote:
> In fact, I am working on the hadoop-connector, so I should create a
> HDFSInput that implements TsFileInput interface for reading tsfile from
> HDFS.
>
> And actually, in DefaultTsFileInput, this method is unsupported which you
> can see in the following.
>
> @Override
> public int read(byte[] b, int off, int len) throws IOException {
> throw new UnsupportedOperationException();
> }
>
> Not all read methods in DefaultTsFileInput modify the position, such as
> 'int read(ByteBuffer dst, long position)’.
>
> So it seems that if you specify the position where you want to start in
> the method signature, the method shouldn’t modify the position of
> TsFileInput which is consistent with FileChannel’s interface.
>
>
> Best,
> --
> Yuan Tian
> School of Software, Tsinghua University
>
> 田原
> 清华大学 软件学院
>
> > 在 2019年9月9日，下午11:19，Boris Zhu 写道：
> >
> > Hi, tian
> > DefaultTsFileInput is a class to implement TsFileInput
> > interface. DefaultTsFileInput's method read actually change the position
> of
> > itself because it uses java.io.FileChannel, and we can use a "position"
> > method to change its position back. But I'm also confusing that why can't
> > change the position of TsFileInput or modify this TsFileInput.
> >
> > Best,
> > Boris zhu
> >
> > On Mon, Sep 9, 2019 at 10:55 PM Yuan Tian
> wrote:
> >
> >> Hi,
> >>
> >> While implementing one read method(shown in the following) in
> TsFileInput
> >> interface, I don’t know whether to modify the position of TsFileInput.
> >>
> >> /**
> >> * read an array of byte from the Input.
> >> *
> >> * @param b -array of byte
> >> * @param off -offset of the Input
> >> * @param len -length
> >> */
> >> int read(byte[] b, int off, int len) throws IOException;
> >>
> >>
> >> I think this method should not modify this TsFileInput and it should be
> >> specified in the comments.
> >>
> >> Best,
> >> --
> >> Yuan Tian
> >> School of Software, Tsinghua University
> >>
> >> 田原
> >> 清华大学 软件学院
> >>
> >>
>
>
Re: Confused about one read method in TsFileInput interface
In fact, I am working on the hadoop-connector, so I should create a HDFSInput
that implements TsFileInput interface for reading tsfile from HDFS.
And actually, in DefaultTsFileInput, this method is unsupported which you can
see in the following.
@Override
public int read(byte[] b, int off, int len) throws IOException {
throw new UnsupportedOperationException();
}
Not all read methods in DefaultTsFileInput modify the position, such as 'int
read(ByteBuffer dst, long position)’.
So it seems that if you specify the position where you want to start in the
method signature, the method shouldn’t modify the position of TsFileInput which
is consistent with FileChannel’s interface.
Best,
--
Yuan Tian
School of Software, Tsinghua University
田原
清华大学 软件学院
> 在 2019年9月9日，下午11:19，Boris Zhu 写道：
>
> Hi, tian
> DefaultTsFileInput is a class to implement TsFileInput
> interface. DefaultTsFileInput's method read actually change the position of
> itself because it uses java.io.FileChannel, and we can use a "position"
> method to change its position back. But I'm also confusing that why can't
> change the position of TsFileInput or modify this TsFileInput.
>
> Best,
> Boris zhu
>
> On Mon, Sep 9, 2019 at 10:55 PM Yuan Tian wrote:
>
>> Hi,
>>
>> While implementing one read method(shown in the following) in TsFileInput
>> interface, I don’t know whether to modify the position of TsFileInput.
>>
>> /**
>> * read an array of byte from the Input.
>> *
>> * @param b -array of byte
>> * @param off -offset of the Input
>> * @param len -length
>> */
>> int read(byte[] b, int off, int len) throws IOException;
>>
>>
>> I think this method should not modify this TsFileInput and it should be
>> specified in the comments.
>>
>> Best,
>> --
>> Yuan Tian
>> School of Software, Tsinghua University
>>
>> 田原
>> 清华大学 软件学院
>>
>>
Re: Confused about one read method in TsFileInput interface
Hi, tian DefaultTsFileInput is a class to implement TsFileInput interface. DefaultTsFileInput's method read actually change the position of itself because it uses java.io.FileChannel, and we can use a "position" method to change its position back. But I'm also confusing that why can't change the position of TsFileInput or modify this TsFileInput. Best, Boris zhu On Mon, Sep 9, 2019 at 10:55 PM Yuan Tian wrote: > Hi, > > While implementing one read method(shown in the following) in TsFileInput > interface, I don’t know whether to modify the position of TsFileInput. > > /** > * read an array of byte from the Input. > * > * @param b -array of byte > * @param off -offset of the Input > * @param len -length > */ > int read(byte[] b, int off, int len) throws IOException; > > > I think this method should not modify this TsFileInput and it should be > specified in the comments. > > Best, > -- > Yuan Tian > School of Software, Tsinghua University > > 田原 > 清华大学 软件学院 > >
