Re: [net] FTPClient doesn't download full images

[Erwin Hogeweg]

Hi,

I have no idea if this is related but I seem to remember from a life long, long 
ago that you can set up and FTP connection in two modes ASCII and binary. The 
default is (used to be) ASCII and when you feed  binary data through that 
connection then (some) non-ascii characters were translated.

Again, I may be talking complete nonsense here but it may be worth to check it 
out.

Cheers,

Erwin


> On Jun 4, 2020, at 09:29, Julia Ruzicka  wrote:
> 
> @Robert Paasche
> 
> The code is used in an AsyncTask and there's more error handling to it 
> (including proper error messages,...), I just simplified it. ;)
> 
> Have you ever tested it with images too? Text files always seem to download 
> properly (no matter what size), it's just the images that are messed up.
> 
> Thanks for the code snippet, I just ran my app with it multiple times and the 
> files are all the right size, even the images, but no program is able to 
> actually open them, so not sure what's going on with that now:
> 
> if(singleFile.length > 0) { //check if file exists
>String localPath = localFolder + File.separator + filename;
>FTPFile single = singleFile[0];
>InputStream inStream = ftpClient.retrieveFileStream(filename);
>ByteArrayOutputStream buffer = new ByteArrayOutputStream();
> 
>int nRead;
>byte[] data = new byte[(int) (single.getSize())]; //length was "*2" but 
> why?
> 
>while ((nRead = inStream.read(data, 0, data.length)) != -1) {
>buffer.write(data, 0, nRead);
>}
> 
>buffer.flush();
>inStream.close();
> 
>FileOutputStream fos = new FileOutputStream(localPath);
>fos.write(data);
>fos.flush();
>fos.close();
> 
>if(ftpClient.completePendingCommand()) {
>Log.d(TAG,"DONE");
>} else {
>Log.d(TAG,"NOT DONE");
>}
> } 
> 
> 
> 
> -
> To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
> For additional commands, e-mail: user-h...@commons.apache.org
> 


-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org

Re: commons-vfs-2.4.1 - Question re. usage of VFS.getManager() - SOLVED

[Erwin Hogeweg]

Hi Bernd,

Thanks for your reply. When I tried this again today 

VFS.getManager().getSchemes()

returns the following (impressive) list of schemes:

[zip, par, ftps, res, ftp, sar, war, file, gz, tmp, http4s, ear, ejb3, 
jar, sftp, http4, ram]

and 

VFS.getManager().resolveFile("ftp://ftp.gnu.org/README;, opts);

works as expected. So I don’t know what happened but it looks like I spent a 
lot of time chasing a ghost. Oh well, at least I learned a thing or two.

Thanks for your help anyway.


Erwin




> 
> I think the problem in your case is that the default filesystem manager does 
> not know about the ftp: scheme. The error message is a bit missleading (since 
> a fix done, probably we should file a bug for this).
> 
> The default manager does try to load the FtpProvider and tries to register it 
> with ftp: so I assume loading of the provider failed. Maybe you are missing 
> dependencies like commons-net (see the website for the dependencies per 
> provider).
> 
> For debugging you can print manager.getSchemes() to see the available schemes 
> (providers).
> Gruss
> Bernd
> 
> 
> --
> http://bernd.eckenfels.net
> 
> 
> Von: Erwin Hogeweg 
> Gesendet: Mittwoch, Oktober 30, 2019 3:29 AM
> An: Commons Users List
> Betreff: commons-vfs-2.4.1 - Question re. usage of VFS.getManager()
> 
> All -
> 
> I am trying to set up a VFS but I struggled a while to get past a 
> FileSystemException. After looking at the VFS source code I understood what 
> was going on, and I was able to create a work-around. However, I am not sure 
> that what I hacked together is the right approach so I am looking for some 
> guidance from the experts here…
> 
> This is what I was trying to do:
> 
> FileObject path = VFS.getManager().resolveFile("ftp://ftp.gnu.org/README;, 
> new FileSystemOptions());
> 
> The issue is, or at least appears to be, that the FileSystemManager doesn’t 
> have a baseFile so the resolveFile blows up with this exception:
> 
> org.apache.commons.vfs2.FileSystemException: Could not find file with URI 
> "ftp://ftp.gnu.org/README; because it is a relative path, and no base URI was 
> provided.
> at 
> org.apache.commons.vfs2.FileSystemException.requireNonNull(FileSystemException.java:87)
> 
> I could not find how/where to define that baseFile so I eventually settled on 
> this work-around:
> 
> StandardFileSystemManager fsManager = (StandardFileSystemManager) 
> VFS.getManager(); // YUCK!
> fsManager.setBaseFile(new File("")); // Another YUCK.
> FileObject path = fsManager.resolveFile("ftp://ftp.gnu.org/README;, opts);
> 
> That works, and now I end up with the correct (expected) path. It doesn’t 
> feel right though so any advice would be greatly appreciated.
> 
> 
> Cheers,
> 
> Erwin
> 
> 

commons-vfs-2.4.1 - Question re. usage of VFS.getManager()

[Erwin Hogeweg]

All -

I am trying to set up a VFS but I struggled a while to get past a 
FileSystemException. After looking at the VFS source code I understood what was 
going on, and I was able to create a work-around. However, I am not sure that 
what I hacked together is the right approach so I am looking for some guidance 
from the experts here…

This is what I was trying to do:

FileObject path = VFS.getManager().resolveFile("ftp://ftp.gnu.org/README;, 
new FileSystemOptions());

The issue is, or at least appears to be, that the FileSystemManager doesn’t 
have a baseFile so the resolveFile blows up with this exception:

org.apache.commons.vfs2.FileSystemException: Could not find file with URI 
"ftp://ftp.gnu.org/README; because it is a relative path, and no base URI was 
provided.
at 
org.apache.commons.vfs2.FileSystemException.requireNonNull(FileSystemException.java:87)

I could not find how/where to define that baseFile so I eventually settled on 
this work-around:

   StandardFileSystemManager fsManager = (StandardFileSystemManager) 
VFS.getManager(); // YUCK!
   fsManager.setBaseFile(new File("")); // Another YUCK.
   FileObject path = fsManager.resolveFile("ftp://ftp.gnu.org/README;, opts);

That works, and now I end up with the correct (expected) path. It doesn’t feel 
right though so any advice would be greatly appreciated.


Cheers,

Erwin

Re: org.apache.commons.lang3.time.StopWatch resolution

[Erwin Hogeweg]

Thanks Bernd,

See below. 

> A 10ms sleep is problematic as a test case since some OS only allow Worse 
> Timer Resolution (for some like Windows it even depends on which timer is 
> currently active, the default timer uses 15,6ms which is only changed in 
> latest Windows 10 I think). So the variation you see is more likely caused by 
> the delay you use and less likely caused by System.nanoTime (as used by 
> Stopwatch).
Interesting. Not sure about that though because I log the 
System.currentTimeMillis around the sleep and the diff is always 10 or more 
never less. 

> For your test (not sure how valuable that is) I would go with 100ms sleep and 
> allow 85-115ms stopwatch results.
I’ll give that a try. 

Thanks again,

Erwin. 
> 
> 
> --
> http://bernd.eckenfels.net
> 
> ________
> Von: Erwin Hogeweg 
> Gesendet: Freitag, Juni 7, 2019 2:21 PM
> An: Commons Users List
> Betreff: Re: org.apache.commons.lang3.time.StopWatch resolution
> 
> Thanks for the replies and the links gents. Still confused. I can understand 
> that you can’t expect nanosecond resolution but the diff I noticed is 1-6 
> ms... on a 10 ms interval. Those are eternities in modern day CPUs.
> 
> I am planning to use the stopwatch with a second resolution so this test is 
> kinda irrelevant but I am still surprised by the result. I might take a look 
> at the implementation later to get a better understanding.
> 
> Thanks again for the help.
> 
> 
> Erwin
> 
> El jun. 6, 2019, a la(s) 22:41, Remko Popma  escribió:
> 
>> Timers and elapsed time in Java is an interesting topic. Can be a moving 
>> target though...
>> 
>> Some recent articles:
>> https://hazelcast.com/blog/locksupport-parknanos-under-the-hood-and-the-curious-case-of-parking/
>> 
>> https://hazelcast.com/blog/locksupport-parknanos-under-the-hood-and-the-curious-case-of-parking-part-ii-windows/
>> 
>> Remko.
>> 
>> (Shameless plug) Every java main() method deserves http://picocli.info
>> 
>>> On Jun 7, 2019, at 6:13, Gary Gregory  wrote:
>>> 
>>> The OS' clock resolution is in play here, which depending on your OS will
>>> give you varying results. Also, on Java 9, you get better clock resolution
>>> for certain APIs. Kinda messy...
>>> 
>>> Gary
>>> 
>>> On Thu, Jun 6, 2019 at 4:28 PM Erwin Hogeweg 
>>> wrote:
>>> 
>>>> Hi,
>>>> 
>>>> I am tad confused about the StopWatch resolution. I have a very basic
>>>> JUnit test that starts a stopwatch, wait for 10ms and then stops it, and
>>>> checks the value. In about 40% of the cases it is less than the 10ms wait
>>>> time.
>>>> 
>>>> Is that expected? What is my blind spot?
>>>> 
>>>> 
>>>> Regards,
>>>> 
>>>> Erwin
>>>> -
>>>> To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
>>>> For additional commands, e-mail: user-h...@commons.apache.org
>>>> 
>>>> 
> 
> 
> -
> To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
> For additional commands, e-mail: user-h...@commons.apache.org
> 


-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org

Re: org.apache.commons.lang3.time.StopWatch resolution

[Erwin Hogeweg]

Thanks for the replies and the links gents. Still confused. I can understand 
that you can’t expect nanosecond resolution but the diff I noticed is 1-6 ms... 
on a 10 ms interval. Those are eternities in modern day CPUs.

I am planning to use the stopwatch with a second resolution so this test is 
kinda irrelevant but I am still surprised by the result. I might take a look at 
the implementation later to get a better understanding. 

Thanks again for the help.


Erwin

El jun. 6, 2019, a la(s) 22:41, Remko Popma  escribió:

> Timers and elapsed time in Java is an interesting topic. Can be a moving 
> target though...
> 
> Some recent articles:
> https://hazelcast.com/blog/locksupport-parknanos-under-the-hood-and-the-curious-case-of-parking/
> 
> https://hazelcast.com/blog/locksupport-parknanos-under-the-hood-and-the-curious-case-of-parking-part-ii-windows/
> 
> Remko.
> 
> (Shameless plug) Every java main() method deserves http://picocli.info
> 
>> On Jun 7, 2019, at 6:13, Gary Gregory  wrote:
>> 
>> The OS' clock resolution is in play here, which depending on your OS will
>> give you varying results. Also, on Java 9, you get better clock resolution
>> for certain APIs. Kinda messy...
>> 
>> Gary
>> 
>> On Thu, Jun 6, 2019 at 4:28 PM Erwin Hogeweg 
>> wrote:
>> 
>>> Hi,
>>> 
>>> I am tad confused about the StopWatch resolution. I have a very basic
>>> JUnit test that starts a stopwatch, wait for 10ms and then stops it, and
>>> checks the value. In about 40% of the cases it is less than the 10ms wait
>>> time.
>>> 
>>> Is that expected? What is my blind spot?
>>> 
>>> 
>>> Regards,
>>> 
>>> Erwin
>>> -
>>> To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
>>> For additional commands, e-mail: user-h...@commons.apache.org
>>> 
>>> 


-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org

org.apache.commons.lang3.time.StopWatch resolution

[Erwin Hogeweg]

Hi,

I am tad confused about the StopWatch resolution. I have a very basic JUnit 
test that starts a stopwatch, wait for 10ms and then stops it, and checks the 
value. In about 40% of the cases it is less than the 10ms wait time.

Is that expected? What is my blind spot?


Regards,

Erwin
-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org

Re: [DBCP] Connection just obtained from datasource is invalid

[Erwin Hogeweg]

 any corpses handed out to
borrowers.  Also make sure that there are enough idle instances in
the pool for the evictor to visit.  For that, you probably want to
vary the borrowing load.  You can set up jmx to observe the pool
stats to see how many are idle at a given time or just log it using
the getNumIdle.  A quick look at the existing pool2 test cases does
not show exactly that scenario covered, so it would be good to add
in any case.

Phil



Thanks,
Shawn


-
To unsubscribe, e-mail: 
user-unsubscr...@commons.apache.org<mailto:user-unsubscr...@commons.apache.org>
For additional commands, e-mail: 
user-h...@commons.apache.org<mailto:user-h...@commons.apache.org>




-
To unsubscribe, e-mail: 
user-unsubscr...@commons.apache.org<mailto:user-unsubscr...@commons.apache.org>
For additional commands, e-mail: 
user-h...@commons.apache.org<mailto:user-h...@commons.apache.org>

Erwin Hogeweg
CTO
3690 Airport Road
Boca Raton, FL 33431
P. +1 (954) 556-6565
M. +1 (561) 306-7395
F. +1 (561) 948-2730
[Seecago]<http://www.seecago.com>

Re: Can't get MySQL non-jta connection pool with Aries-2.4.0 and EL-2.6.2 working - Looking for (more) example(s).

[Erwin Hogeweg]

Hi Tim,

> Have you considered using Aries Transaction Control?
I definitely have, that was going to be my next step once I got this working. I 
just can’t stand that I can’t figure this out. It is not rocket science IMHO ;-)

> It's typically much simpler to configure than the raw JDBC service, and it 
> definitely gives you connection pooling (again, without extra moving parts).
Thanks. I might bite the bullet and skip ahead to Aries Transactions. Would 
have been nice to have a working platform as baseline.

Erwin

> 
> Best Regards,
> 
> Tim Ward
> 
> Sent from my iPhone
> 
> On 24 Jul 2016, at 21:51, Erwin Hogeweg 
> <erwin.hoge...@me.com<mailto:erwin.hoge...@me.com>> wrote:
> 
> Hi,
> 
> Not sure if this is a question for these lists or for the EL list but I 
> figure I start here. Feel free to redirect when you feel it doesn’t belong 
> here.
> 
> I am trying to get a non-jta connection pool (internal connection pool) 
> working with EL 2.6.2, Aries 2.4.0 (incl. EL adapter), dbcp2-2.1 and mySQL, 
> but I must be missing something because I just can’t get it to work properly. 
> Everything works just fine w/o a connection pool, so this is definitely the 
> source of the misery.
> 
> Been struggling with this for a while now, and I am running out of ideas. I 
> think I could use some sample code to point me in the right direction that 
> doesn't use Blueprint? I found some of Christian’s examples, but I don’t 
> think they are using connection pools.
> 
> Below a short summary of what I run into.
> 
> When I am using the ‘original’ MysqlDataSource...
> 
>private DataSource createMySQLDataSource( Dictionary<String, String> 
> dbConnProps ) {
>MysqlDataSource ds = new MysqlDataSource();
>ds.setUrl( dbConnProps.get( "jdbc_url" ) );
>ds.setUser( dbConnProps.get( "jdbc_user" ) );
>ds.setPassword( dbConnProps.get( "jdbc_password" ) );
>return ds;
>}
> 
> … everything kinda works normally. The DataSource, PersistenceProvider and 
> EntityManagerFactory are all created and registered correctly;
> 
> g! services javax.sql.DataSource
> {javax.sql.DataSource}={eclipselink.target-database=MySQL, 
> osgi.jndi.service.name=jdbc/mynonjta, service.id=139, service.bundleid=104, 
> service.scope=singleton}
>  "Registered by bundle:" 
> com.my.project.persistence.mysqldatasource_4.0.0.SNAPSHOT [104]
> 
> g! services javax.persistence.EntityManagerFactory
> {javax.persistence.EntityManagerFactory}={osgi.unit.version=4.0.0.SNAPSHOT, 
> osgi.unit.name=my.pu, 
> osgi.unit.provider=org.eclipse.persistence.jpa.PersistenceProvider, 
> service.id=142, service.bundleid=98, service.scope=singleton}
>  "Registered by bundle:" com.my.project.model_4.0.0.SNAPSHOT [98]
> 
> The performance is horrible though as I don’t really seem to get a connection 
> pool. The connection is closed after every query. On top of that I loose all 
> network connections every few seconds with a:
> 
> com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link 
> failure
> 
> Which has me puzzled for a while now.
> 
> So my next attempt was to use the org.apache.commons.dbcp2.BasicDataSource:
> 
>   private DataSource createMySQLDataSource(  Dictionary<String, String> 
> dbConnProps ) {
> 
>BasicDataSource basicDataSource = new BasicDataSource();
>basicDataSource.setDriverClassName("com.mysql.jdbc.Driver");
> ...
>return basicDataSource;
>}
> 
> This fails because the following exception:
> 
> [EL Severe]: 2016-07-24 
> 14:41:55.872--java.lang.UnsupportedOperationException: Not supported by 
> BasicDataSource
> at 
> org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1552)
> 
> Which is this method:
> 
>@Override
>public Connection getConnection(String user, String pass) throws 
> SQLException {
>// This method isn't supported by the PoolingDataSource returned by
>// the createDataSource
>throw new UnsupportedOperationException("Not supported by 
> BasicDataSource");
>}
> 
> So I figured I create a version with a PoolingDataSource  (following the 
> PoolingDataSourceExample in svn):
> 
>ConnectionFactory connectionFactory = new 
> DriverManagerConnectionFactory(dbConnProps.get( "jdbc_url" ), "user", 
> "password");
>PoolableConnectionFactory poolableConnectionFactory = new 
> PoolableConnectionFactory(connectionFactory, null);
>ObjectPool connectionPool = new 
> GenericObjectPool<>(poolableConnectionFactory);
>poolableConn

Can't get MySQL non-jta connection pool with Aries-2.4.0 and EL-2.6.2 working - Looking for (more) example(s).

[Erwin Hogeweg]

Hi,

Not sure if this is a question for these lists or for the EL list but I figure 
I start here. Feel free to redirect when you feel it doesn’t belong here.

I am trying to get a non-jta connection pool (internal connection pool) working 
with EL 2.6.2, Aries 2.4.0 (incl. EL adapter), dbcp2-2.1 and mySQL, but I must 
be missing something because I just can’t get it to work properly. Everything 
works just fine w/o a connection pool, so this is definitely the source of the 
misery.

Been struggling with this for a while now, and I am running out of ideas. I 
think I could use some sample code to point me in the right direction that 
doesn't use Blueprint? I found some of Christian’s examples, but I don’t think 
they are using connection pools.

Below a short summary of what I run into.

When I am using the ‘original’ MysqlDataSource...

private DataSource createMySQLDataSource( Dictionary 
dbConnProps ) {
MysqlDataSource ds = new MysqlDataSource();
ds.setUrl( dbConnProps.get( "jdbc_url" ) );
ds.setUser( dbConnProps.get( "jdbc_user" ) );
ds.setPassword( dbConnProps.get( "jdbc_password" ) );
return ds;
}

… everything kinda works normally. The DataSource, PersistenceProvider and 
EntityManagerFactory are all created and registered correctly;

g! services javax.sql.DataSource
{javax.sql.DataSource}={eclipselink.target-database=MySQL, 
osgi.jndi.service.name=jdbc/mynonjta, service.id=139, service.bundleid=104, 
service.scope=singleton}
  "Registered by bundle:" 
com.my.project.persistence.mysqldatasource_4.0.0.SNAPSHOT [104]

g! services javax.persistence.EntityManagerFactory
{javax.persistence.EntityManagerFactory}={osgi.unit.version=4.0.0.SNAPSHOT, 
osgi.unit.name=my.pu, 
osgi.unit.provider=org.eclipse.persistence.jpa.PersistenceProvider, 
service.id=142, service.bundleid=98, service.scope=singleton}
  "Registered by bundle:" com.my.project.model_4.0.0.SNAPSHOT [98]

The performance is horrible though as I don’t really seem to get a connection 
pool. The connection is closed after every query. On top of that I loose all 
network connections every few seconds with a:

com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications 
link failure

Which has me puzzled for a while now.

So my next attempt was to use the org.apache.commons.dbcp2.BasicDataSource:

   private DataSource createMySQLDataSource(  Dictionary 
dbConnProps ) {

BasicDataSource basicDataSource = new BasicDataSource();
basicDataSource.setDriverClassName("com.mysql.jdbc.Driver");
...
return basicDataSource;
}

This fails because the following exception:

[EL Severe]: 2016-07-24 14:41:55.872--java.lang.UnsupportedOperationException: 
Not supported by BasicDataSource
at 
org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1552)

Which is this method:

@Override
public Connection getConnection(String user, String pass) throws 
SQLException {
// This method isn't supported by the PoolingDataSource returned by
// the createDataSource
throw new UnsupportedOperationException("Not supported by 
BasicDataSource");
}

So I figured I create a version with a PoolingDataSource  (following the 
PoolingDataSourceExample in svn): 

ConnectionFactory connectionFactory = new 
DriverManagerConnectionFactory(dbConnProps.get( "jdbc_url" ), "user", 
"password");
PoolableConnectionFactory poolableConnectionFactory = new 
PoolableConnectionFactory(connectionFactory, null);
ObjectPool connectionPool = new 
GenericObjectPool<>(poolableConnectionFactory);
poolableConnectionFactory.setPool(connectionPool);
PoolingDataSource dataSource = new 
PoolingDataSource<>(connectionPool);
return dataSource;

But that still gives me an exception:

[EL Severe]: 2016-07-24 16:40:30.392--java.lang.UnsupportedOperationException
at 
org.apache.commons.dbcp2.PoolingDataSource.getConnection(PoolingDataSource.java:156)

So I am kinda lost now.

This is the relevant stuff from the persistence.xml file:


osgi:service/javax.sql.DataSource/(osgi.jndi.service.name=jdbc/mynonjta)


 
 



 
 
 


Although I only see one DataSource registered it somehow feels like there is 
some more stuff going on behind the (EL?) scenes that I don’t have a handle on 
yet.

BTW... I have also created an org.apache.aries.jpa.my.pu.cfg configuration 
file, but when I leave the DB properties out of the persistence.xml I get bunch 
of ClassNotFound exceptions, so that is suspicious.

BTW2… the examples link at the bottom of this page is broken: 
https://commons.apache.org/proper/commons-dbcp/ 



Regards,

Erwin