Re: Different time in Simple History Report

[Karl Wright]

Try it now.

Karl


On Fri, Aug 10, 2018 at 10:57 AM Bisonti Mario 
wrote:

> Yes
>
> sudo ant make-core-deps
>
> Buildfile: /home/administrator/mcfsorce/trunk/build.xml
>
> Trying to override old definition of task javac
>
>
>
> BUILD FAILED
>
> /home/administrator/mcfsorce/trunk/build.xml:2929: taskdef class
> de.thetaphi.forbiddenapis.ant.AntTask cannot be found
>
> using the classloader AntClassLoader[]
>
>
>
> Total time: 0 seconds
>
> administrator@sengvivv01:~/mcfsorce/trunk$
>
>
>
> But the downloaded trunk directory is very small, instead the last trunk
> was bigger:
>
> administrator@sengvivv01:~/mcfsorce$ du -sh tr*
>
> 121Mtrunk
>
> 1.8Gtrunk_19062018
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright 
> *Inviato:* venerdì 10 agosto 2018 16:47
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Different time in Simple History Report
>
>
>
> Did you first do:
>
>
>
> ant make-core-deps
>
>
>
> ?
>
> Karl
>
>
>
>
>
> On Fri, Aug 10, 2018 at 5:04 AM Bisonti Mario 
> wrote:
>
> Thanks Karl.
>
> I tried to compile the trunk version but I obtian:
>
> Buildfile: /home/administrator/mcfsorce/trunk/build.xml
>
> Trying to override old definition of task javac
>
>
>
> BUILD FAILED
>
> /home/administrator/mcfsorce/trunk/build.xml:2929: taskdef class
> de.thetaphi.forbiddenapis.ant.AntTask cannot be found
>
> using the classloader AntClassLoader[]
>
>
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright 
> *Inviato:* venerdì 10 agosto 2018 10:53
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Different time in Simple History Report
>
>
>
> I've committed a change to trunk which will restore the pre-2016 behavior.
>
>
>
> Karl
>
>
>
> On Fri, Aug 10, 2018 at 3:40 AM Karl Wright  wrote:
>
> The code that formats the time is here:
>
>
>
> >>
>
> String startTimeString =
> org.apache.manifoldcf.ui.util.Formatter.formatTime(Converter.asLong(row.getValue("starttime")));
>
> <<
>
>
>
> This explicitly uses UTC as the timezone:
>
> >>
>
>   /** Format a long as an understandable date.
>
>   *@param time is the long.
>
>   *@return the date, as a human-readable string.  This date will be in
> local time.
>
>   */
>
>   public static String formatTime(long time)
>
>   {
>
> Calendar c = new GregorianCalendar(TimeZone.getTimeZone("UTC"),
> Locale.ROOT);
>
> c.setTimeInMillis(time);
>
> // We want to format this string in a compact way:
>
> // mm-dd- hh:mm:ss.mmm
>
> StringBuilder returnString = new StringBuilder();
>
> writechars(returnString,c.get(Calendar.MONTH)+1,2);
>
> returnString.append("-");
>
> writechars(returnString,c.get(Calendar.DAY_OF_MONTH),2);
>
> returnString.append("-");
>
> writechars(returnString,c.get(Calendar.YEAR),4);
>
> returnString.append(" ");
>
> writechars(returnString,c.get(Calendar.HOUR_OF_DAY),2);
>
> returnString.append(":");
>
> writechars(returnString,c.get(Calendar.MINUTE),2);
>
> returnString.append(":");
>
> writechars(returnString,c.get(Calendar.SECOND),2);
>
> returnString.append(".");
>
> writechars(returnString,c.get(Calendar.MILLISECOND),3);
>
> return returnString.toString();
>
>   }
>
> <<
>
> This was last changed:
>
> >>
>
> 1756230kwright Calendar c = new
> GregorianCalendar(TimeZone.getTimeZone("UTC"), Locale.ROOT);
>
> <<
>
>
>
> The reason for the change:
>
> >>
>
> 
>
> r1756230 | kwright | 2016-08-12 18:20:00 -0400 (Fri, 12 Aug 2016) | 1 line
>
>
>
> Fix for CONNECTORS-1332.  Committed on behalf of Furkan KAMACI.
>
> <<
>
> CONNECTORS-1332 is about calling forbidden APIS:
>
>
>
> >>
>
> We should avoid forbidden calls
> 
>  and
> check for it in the ant build.
>
> <<
>
>
>
> The actual change was:
>
> >>
>
> C:\wip\mcf\trunk\framework\ui-core\src\main\java\org\apache\manifoldcf\ui\util>svn
> diff -c 1756230
>
> Index: Formatter.java
>
> ===
>
> --- Formatter.java  (revision 1756229)
>
> +++ Formatter.java  (revision 1756230)
>
> @@ -32,7 +32,7 @@
>
>*/
>
>public static String formatTime(long time)
>
>{
>
> -Calendar c = new GregorianCalendar();
>
> +Calendar c = new GregorianCalendar(TimeZone.getTimeZone("UTC"),
> Locale.ROOT);
>
>  c.setTimeInMillis(time);
>
>  // We want to format this string in a compact way:
>
>  // mm-dd- hh:mm:ss.mmm
>
> <<
>
>
>
>
>
> As you see, formerly the timezone was local time.  The change required an
> explicit timezone in order to pass the forbidden APIs test, and UTC was
> used.
>
> I am happy to try to change this since it's been this way only since 2016,
> 

R: Different time in Simple History Report

[Bisonti Mario]

Yes
sudo ant make-core-deps
Buildfile: /home/administrator/mcfsorce/trunk/build.xml
Trying to override old definition of task javac

BUILD FAILED
/home/administrator/mcfsorce/trunk/build.xml:2929: taskdef class 
de.thetaphi.forbiddenapis.ant.AntTask cannot be found
using the classloader AntClassLoader[]

Total time: 0 seconds
administrator@sengvivv01:~/mcfsorce/trunk$

But the downloaded trunk directory is very small, instead the last trunk was 
bigger:
administrator@sengvivv01:~/mcfsorce$ du -sh tr*
121Mtrunk
1.8Gtrunk_19062018




Da: Karl Wright 
Inviato: venerdì 10 agosto 2018 16:47
A: user@manifoldcf.apache.org
Oggetto: Re: Different time in Simple History Report

Did you first do:

ant make-core-deps

?

Karl


On Fri, Aug 10, 2018 at 5:04 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
Thanks Karl.
I tried to compile the trunk version but I obtian:

Buildfile: /home/administrator/mcfsorce/trunk/build.xml
Trying to override old definition of task javac

BUILD FAILED
/home/administrator/mcfsorce/trunk/build.xml:2929: taskdef class 
de.thetaphi.forbiddenapis.ant.AntTask cannot be found
using the classloader AntClassLoader[]





Da: Karl Wright mailto:daddy...@gmail.com>>
Inviato: venerdì 10 agosto 2018 10:53
A: user@manifoldcf.apache.org
Oggetto: Re: Different time in Simple History Report

I've committed a change to trunk which will restore the pre-2016 behavior.

Karl

On Fri, Aug 10, 2018 at 3:40 AM Karl Wright 
mailto:daddy...@gmail.com>> wrote:
The code that formats the time is here:

>>
String startTimeString = 
org.apache.manifoldcf.ui.util.Formatter.formatTime(Converter.asLong(row.getValue("starttime")));
<<

This explicitly uses UTC as the timezone:

>>
  /** Format a long as an understandable date.
  *@param time is the long.
  *@return the date, as a human-readable string.  This date will be in local 
time.
  */
  public static String formatTime(long time)
  {
Calendar c = new GregorianCalendar(TimeZone.getTimeZone("UTC"), 
Locale.ROOT);
c.setTimeInMillis(time);
// We want to format this string in a compact way:
// mm-dd- hh:mm:ss.mmm
StringBuilder returnString = new StringBuilder();
writechars(returnString,c.get(Calendar.MONTH)+1,2);
returnString.append("-");
writechars(returnString,c.get(Calendar.DAY_OF_MONTH),2);
returnString.append("-");
writechars(returnString,c.get(Calendar.YEAR),4);
returnString.append(" ");
writechars(returnString,c.get(Calendar.HOUR_OF_DAY),2);
returnString.append(":");
writechars(returnString,c.get(Calendar.MINUTE),2);
returnString.append(":");
writechars(returnString,c.get(Calendar.SECOND),2);
returnString.append(".");
writechars(returnString,c.get(Calendar.MILLISECOND),3);
return returnString.toString();
  }
<<

This was last changed:

>>
1756230kwright Calendar c = new 
GregorianCalendar(TimeZone.getTimeZone("UTC"), Locale.ROOT);
<<

The reason for the change:

>>

r1756230 | kwright | 2016-08-12 18:20:00 -0400 (Fri, 12 Aug 2016) | 1 line

Fix for CONNECTORS-1332.  Committed on behalf of Furkan KAMACI.
<<
CONNECTORS-1332 is about calling forbidden APIS:

>>
We should avoid forbidden 
calls
 and check for it in the ant build.
<<

The actual change was:
>>
C:\wip\mcf\trunk\framework\ui-core\src\main\java\org\apache\manifoldcf\ui\util>svn
 diff -c 1756230
Index: Formatter.java
===
--- Formatter.java  (revision 1756229)
+++ Formatter.java  (revision 1756230)
@@ -32,7 +32,7 @@
   */
   public static String formatTime(long time)
   {
-Calendar c = new GregorianCalendar();
+Calendar c = new GregorianCalendar(TimeZone.getTimeZone("UTC"), 
Locale.ROOT);
 c.setTimeInMillis(time);
 // We want to format this string in a compact way:
 // mm-dd- hh:mm:ss.mmm
<<


As you see, formerly the timezone was local time.  The change required an 
explicit timezone in order to pass the forbidden APIs test, and UTC was used.

I am happy to try to change this since it's been this way only since 2016, if I 
can find a way that will not break forbiddenAPIs.

Karl


On Fri, Aug 10, 2018 at 2:42 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
Hallo Karl.
My server timezone is set as the browser timezone (europe/Rome) as you can see, 
but the list is two hour less my time zone.
So, it seems that the list uses the “universal time” instead of time zone

administrator@sengvivv01:~$ timedatectl
  Local time: Fri 2018-08-10 08:39:28 CEST
 

Re: Different time in Simple History Report

[Karl Wright]

Did you first do:

ant make-core-deps

?

Karl


On Fri, Aug 10, 2018 at 5:04 AM Bisonti Mario 
wrote:

> Thanks Karl.
>
> I tried to compile the trunk version but I obtian:
>
> Buildfile: /home/administrator/mcfsorce/trunk/build.xml
>
> Trying to override old definition of task javac
>
>
>
> BUILD FAILED
>
> /home/administrator/mcfsorce/trunk/build.xml:2929: taskdef class
> de.thetaphi.forbiddenapis.ant.AntTask cannot be found
>
> using the classloader AntClassLoader[]
>
>
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright 
> *Inviato:* venerdì 10 agosto 2018 10:53
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Different time in Simple History Report
>
>
>
> I've committed a change to trunk which will restore the pre-2016 behavior.
>
>
>
> Karl
>
>
>
> On Fri, Aug 10, 2018 at 3:40 AM Karl Wright  wrote:
>
> The code that formats the time is here:
>
>
>
> >>
>
> String startTimeString =
> org.apache.manifoldcf.ui.util.Formatter.formatTime(Converter.asLong(row.getValue("starttime")));
>
> <<
>
>
>
> This explicitly uses UTC as the timezone:
>
> >>
>
>   /** Format a long as an understandable date.
>
>   *@param time is the long.
>
>   *@return the date, as a human-readable string.  This date will be in
> local time.
>
>   */
>
>   public static String formatTime(long time)
>
>   {
>
> Calendar c = new GregorianCalendar(TimeZone.getTimeZone("UTC"),
> Locale.ROOT);
>
> c.setTimeInMillis(time);
>
> // We want to format this string in a compact way:
>
> // mm-dd- hh:mm:ss.mmm
>
> StringBuilder returnString = new StringBuilder();
>
> writechars(returnString,c.get(Calendar.MONTH)+1,2);
>
> returnString.append("-");
>
> writechars(returnString,c.get(Calendar.DAY_OF_MONTH),2);
>
> returnString.append("-");
>
> writechars(returnString,c.get(Calendar.YEAR),4);
>
> returnString.append(" ");
>
> writechars(returnString,c.get(Calendar.HOUR_OF_DAY),2);
>
> returnString.append(":");
>
> writechars(returnString,c.get(Calendar.MINUTE),2);
>
> returnString.append(":");
>
> writechars(returnString,c.get(Calendar.SECOND),2);
>
> returnString.append(".");
>
> writechars(returnString,c.get(Calendar.MILLISECOND),3);
>
> return returnString.toString();
>
>   }
>
> <<
>
> This was last changed:
>
> >>
>
> 1756230kwright Calendar c = new
> GregorianCalendar(TimeZone.getTimeZone("UTC"), Locale.ROOT);
>
> <<
>
>
>
> The reason for the change:
>
> >>
>
> 
>
> r1756230 | kwright | 2016-08-12 18:20:00 -0400 (Fri, 12 Aug 2016) | 1 line
>
>
>
> Fix for CONNECTORS-1332.  Committed on behalf of Furkan KAMACI.
>
> <<
>
> CONNECTORS-1332 is about calling forbidden APIS:
>
>
>
> >>
>
> We should avoid forbidden calls
> 
>  and
> check for it in the ant build.
>
> <<
>
>
>
> The actual change was:
>
> >>
>
> C:\wip\mcf\trunk\framework\ui-core\src\main\java\org\apache\manifoldcf\ui\util>svn
> diff -c 1756230
>
> Index: Formatter.java
>
> ===
>
> --- Formatter.java  (revision 1756229)
>
> +++ Formatter.java  (revision 1756230)
>
> @@ -32,7 +32,7 @@
>
>*/
>
>public static String formatTime(long time)
>
>{
>
> -Calendar c = new GregorianCalendar();
>
> +Calendar c = new GregorianCalendar(TimeZone.getTimeZone("UTC"),
> Locale.ROOT);
>
>  c.setTimeInMillis(time);
>
>  // We want to format this string in a compact way:
>
>  // mm-dd- hh:mm:ss.mmm
>
> <<
>
>
>
>
>
> As you see, formerly the timezone was local time.  The change required an
> explicit timezone in order to pass the forbidden APIs test, and UTC was
> used.
>
> I am happy to try to change this since it's been this way only since 2016,
> if I can find a way that will not break forbiddenAPIs.
>
>
>
> Karl
>
>
>
>
>
> On Fri, Aug 10, 2018 at 2:42 AM Bisonti Mario 
> wrote:
>
> Hallo Karl.
>
> My server timezone is set as the browser timezone (europe/Rome) as you can
> see, but the list is two hour less my time zone.
>
> So, it seems that the list uses the “universal time” instead of time zone
>
>
>
> administrator@sengvivv01:~$ timedatectl
>
>   Local time: Fri 2018-08-10 08:39:28 CEST
>
>   Universal time: Fri 2018-08-10 06:39:28 UTC
>
> RTC time: Fri 2018-08-10 06:39:28
>
>Time zone: Europe/Rome (CEST, +0200)
>
>System clock synchronized: yes
>
> systemd-timesyncd.service active: yes
>
>  RTC in local TZ: no
>
>
>
>
>
> What could I do?
>
> Thanks a lot
>
>
>
>
>
>
>
> *Da:* Karl Wright 
> *Inviato:* 

Re: crawl interrupted

[Gustavo Beneitez]

I see.
Starting another process somehow it turns out to the same point

DEBUG 2018-08-10T10:31:08,336 (Thread-38685) - Cancelling request execution

No one  is pushing the button through the Web User Interface, so I guess I
have to look at the source code.



El vie., 10 ago. 2018 a las 10:55, Karl Wright ()
escribió:

> There is no configuration I know of that is related to this.
>
> The connection manager being shut down occurs when the agents process is
> being shut down or killed.  This is not something that MCF will do except
> when told to.
>
> Karl
>
>
> On Fri, Aug 10, 2018 at 4:28 AM Gustavo Beneitez <
> gustavo.benei...@gmail.com> wrote:
>
>> Hi again Karl,
>>
>> I've been further investigating this issue since today it happened again.
>> I was able to capture the instant when jobs stopped, It seems they receive
>> an abort command.
>> I then investigate the logs (they are centralised with Kibana) and the
>> most strange thing I can see is this log:
>>
>>  DEBUG 2018-08-10T09:58:16,593 (Agents idle cleanup thread) - Connection
>> manager shut down
>>
>> Who can decide this kind of behaviour? Is there any configuration related
>> to this?
>>
>> Thanks!
>>
>> El jue., 9 ago. 2018 a las 13:08, Gustavo Beneitez (<
>> gustavo.benei...@gmail.com>) escribió:
>>
>>> Yes, you are right, maybe I can execute OPTIMIZE queries if job fails
>>> again, let's just cross fingers.
>>>
>>> El jue., 9 ago. 2018 a las 12:28, Karl Wright ()
>>> escribió:
>>>
 There is no autovacuum for MySQL.  MySQL apparently does dead tuple
 cleanup as it goes.

 Karl

 On Thu, Aug 9, 2018 at 6:13 AM Gustavo Beneitez <
 gustavo.benei...@gmail.com> wrote:

> Hi,
>
> looking at the manifoldCF pom I can see
>
> 1.0.4-SNAPSHOT
>
> I'm not aware of any change in database, in fact ours is MySQL, I
> don't know if "auto_vacuum" property is present in MySQL installation.
>
> Thanks!
>
> El jue., 9 ago. 2018 a las 11:19, msaunier ()
> escribió:
>
>> Hi Gustavo,
>>
>>
>>
>> What is your ManifoldCF version?
>>
>> Do you have disabled auto_vacuum on your SQL configuration?
>>
>>
>>
>> Maxence,
>>
>>
>>
>>
>>
>>
>>
>> *De :* Gustavo Beneitez [mailto:gustavo.benei...@gmail.com]
>> *Envoyé :* jeudi 9 août 2018 11:17
>> *À :* user@manifoldcf.apache.org
>> *Objet :* crawl interrupted
>>
>>
>>
>> Hi all,
>>
>>
>>
>> The Manifold crawler just aborted his jobs and recordeda message in
>> job status:
>>
>>
>>
>>  Error: Unexpected jobqueue status - record id 1533799203323,
>> expecting active status, saw 2
>>
>>
>>
>> Do you know what does it mean? Maybe I have to ask for GC or catalina
>> logs.
>>
>>
>>
>> Thanks!
>>
>

R: Different time in Simple History Report

[Bisonti Mario]

Thanks Karl.
I tried to compile the trunk version but I obtian:

Buildfile: /home/administrator/mcfsorce/trunk/build.xml
Trying to override old definition of task javac

BUILD FAILED
/home/administrator/mcfsorce/trunk/build.xml:2929: taskdef class 
de.thetaphi.forbiddenapis.ant.AntTask cannot be found
using the classloader AntClassLoader[]





Da: Karl Wright 
Inviato: venerdì 10 agosto 2018 10:53
A: user@manifoldcf.apache.org
Oggetto: Re: Different time in Simple History Report

I've committed a change to trunk which will restore the pre-2016 behavior.

Karl

On Fri, Aug 10, 2018 at 3:40 AM Karl Wright 
mailto:daddy...@gmail.com>> wrote:
The code that formats the time is here:

>>
String startTimeString = 
org.apache.manifoldcf.ui.util.Formatter.formatTime(Converter.asLong(row.getValue("starttime")));
<<

This explicitly uses UTC as the timezone:

>>
  /** Format a long as an understandable date.
  *@param time is the long.
  *@return the date, as a human-readable string.  This date will be in local 
time.
  */
  public static String formatTime(long time)
  {
Calendar c = new GregorianCalendar(TimeZone.getTimeZone("UTC"), 
Locale.ROOT);
c.setTimeInMillis(time);
// We want to format this string in a compact way:
// mm-dd- hh:mm:ss.mmm
StringBuilder returnString = new StringBuilder();
writechars(returnString,c.get(Calendar.MONTH)+1,2);
returnString.append("-");
writechars(returnString,c.get(Calendar.DAY_OF_MONTH),2);
returnString.append("-");
writechars(returnString,c.get(Calendar.YEAR),4);
returnString.append(" ");
writechars(returnString,c.get(Calendar.HOUR_OF_DAY),2);
returnString.append(":");
writechars(returnString,c.get(Calendar.MINUTE),2);
returnString.append(":");
writechars(returnString,c.get(Calendar.SECOND),2);
returnString.append(".");
writechars(returnString,c.get(Calendar.MILLISECOND),3);
return returnString.toString();
  }
<<

This was last changed:

>>
1756230kwright Calendar c = new 
GregorianCalendar(TimeZone.getTimeZone("UTC"), Locale.ROOT);
<<

The reason for the change:

>>

r1756230 | kwright | 2016-08-12 18:20:00 -0400 (Fri, 12 Aug 2016) | 1 line

Fix for CONNECTORS-1332.  Committed on behalf of Furkan KAMACI.
<<
CONNECTORS-1332 is about calling forbidden APIS:

>>
We should avoid forbidden 
calls
 and check for it in the ant build.
<<

The actual change was:
>>
C:\wip\mcf\trunk\framework\ui-core\src\main\java\org\apache\manifoldcf\ui\util>svn
 diff -c 1756230
Index: Formatter.java
===
--- Formatter.java  (revision 1756229)
+++ Formatter.java  (revision 1756230)
@@ -32,7 +32,7 @@
   */
   public static String formatTime(long time)
   {
-Calendar c = new GregorianCalendar();
+Calendar c = new GregorianCalendar(TimeZone.getTimeZone("UTC"), 
Locale.ROOT);
 c.setTimeInMillis(time);
 // We want to format this string in a compact way:
 // mm-dd- hh:mm:ss.mmm
<<


As you see, formerly the timezone was local time.  The change required an 
explicit timezone in order to pass the forbidden APIs test, and UTC was used.

I am happy to try to change this since it's been this way only since 2016, if I 
can find a way that will not break forbiddenAPIs.

Karl


On Fri, Aug 10, 2018 at 2:42 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
Hallo Karl.
My server timezone is set as the browser timezone (europe/Rome) as you can see, 
but the list is two hour less my time zone.
So, it seems that the list uses the “universal time” instead of time zone

administrator@sengvivv01:~$ timedatectl
  Local time: Fri 2018-08-10 08:39:28 CEST
  Universal time: Fri 2018-08-10 06:39:28 UTC
RTC time: Fri 2018-08-10 06:39:28
   Time zone: Europe/Rome (CEST, +0200)
   System clock synchronized: yes
systemd-timesyncd.service active: yes
 RTC in local TZ: no


What could I do?
Thanks a lot



Da: Karl Wright mailto:daddy...@gmail.com>>
Inviato: giovedì 9 agosto 2018 21:36
A: user@manifoldcf.apache.org
Oggetto: Re: Different time in Simple History Report

Hi Mario.

The pulldown allows you to select times based on the current (browser) time 
zone.

The display is in *server* timezone.  That accounts for the difference.

Karl


On Thu, Aug 9, 2018 at 10:23 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
Hallo
I see a difference from the start time in “Simple History Report”

It seems late of 

Re: crawl interrupted

[Karl Wright]

There is no configuration I know of that is related to this.

The connection manager being shut down occurs when the agents process is
being shut down or killed.  This is not something that MCF will do except
when told to.

Karl


On Fri, Aug 10, 2018 at 4:28 AM Gustavo Beneitez 
wrote:

> Hi again Karl,
>
> I've been further investigating this issue since today it happened again.
> I was able to capture the instant when jobs stopped, It seems they receive
> an abort command.
> I then investigate the logs (they are centralised with Kibana) and the
> most strange thing I can see is this log:
>
>  DEBUG 2018-08-10T09:58:16,593 (Agents idle cleanup thread) - Connection
> manager shut down
>
> Who can decide this kind of behaviour? Is there any configuration related
> to this?
>
> Thanks!
>
> El jue., 9 ago. 2018 a las 13:08, Gustavo Beneitez (<
> gustavo.benei...@gmail.com>) escribió:
>
>> Yes, you are right, maybe I can execute OPTIMIZE queries if job fails
>> again, let's just cross fingers.
>>
>> El jue., 9 ago. 2018 a las 12:28, Karl Wright ()
>> escribió:
>>
>>> There is no autovacuum for MySQL.  MySQL apparently does dead tuple
>>> cleanup as it goes.
>>>
>>> Karl
>>>
>>> On Thu, Aug 9, 2018 at 6:13 AM Gustavo Beneitez <
>>> gustavo.benei...@gmail.com> wrote:
>>>
 Hi,

 looking at the manifoldCF pom I can see

 1.0.4-SNAPSHOT

 I'm not aware of any change in database, in fact ours is MySQL, I don't
 know if "auto_vacuum" property is present in MySQL installation.

 Thanks!

 El jue., 9 ago. 2018 a las 11:19, msaunier ()
 escribió:

> Hi Gustavo,
>
>
>
> What is your ManifoldCF version?
>
> Do you have disabled auto_vacuum on your SQL configuration?
>
>
>
> Maxence,
>
>
>
>
>
>
>
> *De :* Gustavo Beneitez [mailto:gustavo.benei...@gmail.com]
> *Envoyé :* jeudi 9 août 2018 11:17
> *À :* user@manifoldcf.apache.org
> *Objet :* crawl interrupted
>
>
>
> Hi all,
>
>
>
> The Manifold crawler just aborted his jobs and recordeda message in
> job status:
>
>
>
>  Error: Unexpected jobqueue status - record id 1533799203323,
> expecting active status, saw 2
>
>
>
> Do you know what does it mean? Maybe I have to ask for GC or catalina
> logs.
>
>
>
> Thanks!
>

Re: Different time in Simple History Report

[Karl Wright]

I've committed a change to trunk which will restore the pre-2016 behavior.

Karl

On Fri, Aug 10, 2018 at 3:40 AM Karl Wright  wrote:

> The code that formats the time is here:
>
> >>
> String startTimeString =
> org.apache.manifoldcf.ui.util.Formatter.formatTime(Converter.asLong(row.getValue("starttime")));
> <<
>
> This explicitly uses UTC as the timezone:
>
> >>
>   /** Format a long as an understandable date.
>   *@param time is the long.
>   *@return the date, as a human-readable string.  This date will be in
> local time.
>   */
>   public static String formatTime(long time)
>   {
> Calendar c = new GregorianCalendar(TimeZone.getTimeZone("UTC"),
> Locale.ROOT);
> c.setTimeInMillis(time);
> // We want to format this string in a compact way:
> // mm-dd- hh:mm:ss.mmm
> StringBuilder returnString = new StringBuilder();
> writechars(returnString,c.get(Calendar.MONTH)+1,2);
> returnString.append("-");
> writechars(returnString,c.get(Calendar.DAY_OF_MONTH),2);
> returnString.append("-");
> writechars(returnString,c.get(Calendar.YEAR),4);
> returnString.append(" ");
> writechars(returnString,c.get(Calendar.HOUR_OF_DAY),2);
> returnString.append(":");
> writechars(returnString,c.get(Calendar.MINUTE),2);
> returnString.append(":");
> writechars(returnString,c.get(Calendar.SECOND),2);
> returnString.append(".");
> writechars(returnString,c.get(Calendar.MILLISECOND),3);
> return returnString.toString();
>   }
> <<
>
> This was last changed:
>
> >>
> 1756230kwright Calendar c = new
> GregorianCalendar(TimeZone.getTimeZone("UTC"), Locale.ROOT);
> <<
>
> The reason for the change:
>
> >>
> 
> r1756230 | kwright | 2016-08-12 18:20:00 -0400 (Fri, 12 Aug 2016) | 1 line
>
> Fix for CONNECTORS-1332.  Committed on behalf of Furkan KAMACI.
> <<
>
> CONNECTORS-1332 is about calling forbidden APIS:
>
> >>
> We should avoid forbidden calls
>  and check for it
> in the ant build.
> <<
>
> The actual change was:
>
> >>
> C:\wip\mcf\trunk\framework\ui-core\src\main\java\org\apache\manifoldcf\ui\util>svn
> diff -c 1756230
> Index: Formatter.java
> ===
> --- Formatter.java  (revision 1756229)
> +++ Formatter.java  (revision 1756230)
> @@ -32,7 +32,7 @@
>*/
>public static String formatTime(long time)
>{
> -Calendar c = new GregorianCalendar();
> +Calendar c = new GregorianCalendar(TimeZone.getTimeZone("UTC"),
> Locale.ROOT);
>  c.setTimeInMillis(time);
>  // We want to format this string in a compact way:
>  // mm-dd- hh:mm:ss.mmm
> <<
>
>
> As you see, formerly the timezone was local time.  The change required an
> explicit timezone in order to pass the forbidden APIs test, and UTC was
> used.
>
> I am happy to try to change this since it's been this way only since 2016,
> if I can find a way that will not break forbiddenAPIs.
>
> Karl
>
>
> On Fri, Aug 10, 2018 at 2:42 AM Bisonti Mario 
> wrote:
>
>> Hallo Karl.
>>
>> My server timezone is set as the browser timezone (europe/Rome) as you
>> can see, but the list is two hour less my time zone.
>>
>> So, it seems that the list uses the “universal time” instead of time zone
>>
>>
>>
>> administrator@sengvivv01:~$ timedatectl
>>
>>   Local time: Fri 2018-08-10 08:39:28 CEST
>>
>>   Universal time: Fri 2018-08-10 06:39:28 UTC
>>
>> RTC time: Fri 2018-08-10 06:39:28
>>
>>Time zone: Europe/Rome (CEST, +0200)
>>
>>System clock synchronized: yes
>>
>> systemd-timesyncd.service active: yes
>>
>>  RTC in local TZ: no
>>
>>
>>
>>
>>
>> What could I do?
>>
>> Thanks a lot
>>
>>
>>
>>
>>
>>
>>
>> *Da:* Karl Wright 
>> *Inviato:* giovedì 9 agosto 2018 21:36
>> *A:* user@manifoldcf.apache.org
>> *Oggetto:* Re: Different time in Simple History Report
>>
>>
>>
>> Hi Mario.
>>
>> The pulldown allows you to select times based on the current (browser)
>> time zone.
>>
>> The display is in *server* timezone.  That accounts for the difference.
>>
>>
>>
>> Karl
>>
>>
>>
>>
>>
>> On Thu, Aug 9, 2018 at 10:23 AM Bisonti Mario 
>> wrote:
>>
>> Hallo
>>
>> I see a difference from the start time in “Simple History Report”
>>
>>
>>
>> It seems late of 2 hours.
>>
>>
>>
>> Have I to set timezone for this report?
>>
>>
>>
>> Thanks a lot
>>
>> See the attachment
>>
>>
>>
>>
>>
>>

Re: crawl interrupted

[Gustavo Beneitez]

Hi again Karl,

I've been further investigating this issue since today it happened again. I
was able to capture the instant when jobs stopped, It seems they receive an
abort command.
I then investigate the logs (they are centralised with Kibana) and the most
strange thing I can see is this log:

 DEBUG 2018-08-10T09:58:16,593 (Agents idle cleanup thread) - Connection
manager shut down

Who can decide this kind of behaviour? Is there any configuration related
to this?

Thanks!

El jue., 9 ago. 2018 a las 13:08, Gustavo Beneitez (<
gustavo.benei...@gmail.com>) escribió:

> Yes, you are right, maybe I can execute OPTIMIZE queries if job fails
> again, let's just cross fingers.
>
> El jue., 9 ago. 2018 a las 12:28, Karl Wright ()
> escribió:
>
>> There is no autovacuum for MySQL.  MySQL apparently does dead tuple
>> cleanup as it goes.
>>
>> Karl
>>
>> On Thu, Aug 9, 2018 at 6:13 AM Gustavo Beneitez <
>> gustavo.benei...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> looking at the manifoldCF pom I can see
>>>
>>> 1.0.4-SNAPSHOT
>>>
>>> I'm not aware of any change in database, in fact ours is MySQL, I don't
>>> know if "auto_vacuum" property is present in MySQL installation.
>>>
>>> Thanks!
>>>
>>> El jue., 9 ago. 2018 a las 11:19, msaunier ()
>>> escribió:
>>>
 Hi Gustavo,



 What is your ManifoldCF version?

 Do you have disabled auto_vacuum on your SQL configuration?



 Maxence,







 *De :* Gustavo Beneitez [mailto:gustavo.benei...@gmail.com]
 *Envoyé :* jeudi 9 août 2018 11:17
 *À :* user@manifoldcf.apache.org
 *Objet :* crawl interrupted



 Hi all,



 The Manifold crawler just aborted his jobs and recordeda message in job
 status:



  Error: Unexpected jobqueue status - record id 1533799203323, expecting
 active status, saw 2



 Do you know what does it mean? Maybe I have to ask for GC or catalina
 logs.



 Thanks!

>>>

Re: Different time in Simple History Report

[Karl Wright]

The code that formats the time is here:

>>
String startTimeString =
org.apache.manifoldcf.ui.util.Formatter.formatTime(Converter.asLong(row.getValue("starttime")));
<<

This explicitly uses UTC as the timezone:

>>
  /** Format a long as an understandable date.
  *@param time is the long.
  *@return the date, as a human-readable string.  This date will be in
local time.
  */
  public static String formatTime(long time)
  {
Calendar c = new GregorianCalendar(TimeZone.getTimeZone("UTC"),
Locale.ROOT);
c.setTimeInMillis(time);
// We want to format this string in a compact way:
// mm-dd- hh:mm:ss.mmm
StringBuilder returnString = new StringBuilder();
writechars(returnString,c.get(Calendar.MONTH)+1,2);
returnString.append("-");
writechars(returnString,c.get(Calendar.DAY_OF_MONTH),2);
returnString.append("-");
writechars(returnString,c.get(Calendar.YEAR),4);
returnString.append(" ");
writechars(returnString,c.get(Calendar.HOUR_OF_DAY),2);
returnString.append(":");
writechars(returnString,c.get(Calendar.MINUTE),2);
returnString.append(":");
writechars(returnString,c.get(Calendar.SECOND),2);
returnString.append(".");
writechars(returnString,c.get(Calendar.MILLISECOND),3);
return returnString.toString();
  }
<<

This was last changed:

>>
1756230kwright Calendar c = new
GregorianCalendar(TimeZone.getTimeZone("UTC"), Locale.ROOT);
<<

The reason for the change:

>>

r1756230 | kwright | 2016-08-12 18:20:00 -0400 (Fri, 12 Aug 2016) | 1 line

Fix for CONNECTORS-1332.  Committed on behalf of Furkan KAMACI.
<<

CONNECTORS-1332 is about calling forbidden APIS:

>>
We should avoid forbidden calls
 and check for it
in the ant build.
<<

The actual change was:

>>
C:\wip\mcf\trunk\framework\ui-core\src\main\java\org\apache\manifoldcf\ui\util>svn
diff -c 1756230
Index: Formatter.java
===
--- Formatter.java  (revision 1756229)
+++ Formatter.java  (revision 1756230)
@@ -32,7 +32,7 @@
   */
   public static String formatTime(long time)
   {
-Calendar c = new GregorianCalendar();
+Calendar c = new GregorianCalendar(TimeZone.getTimeZone("UTC"),
Locale.ROOT);
 c.setTimeInMillis(time);
 // We want to format this string in a compact way:
 // mm-dd- hh:mm:ss.mmm
<<


As you see, formerly the timezone was local time.  The change required an
explicit timezone in order to pass the forbidden APIs test, and UTC was
used.

I am happy to try to change this since it's been this way only since 2016,
if I can find a way that will not break forbiddenAPIs.

Karl


On Fri, Aug 10, 2018 at 2:42 AM Bisonti Mario 
wrote:

> Hallo Karl.
>
> My server timezone is set as the browser timezone (europe/Rome) as you can
> see, but the list is two hour less my time zone.
>
> So, it seems that the list uses the “universal time” instead of time zone
>
>
>
> administrator@sengvivv01:~$ timedatectl
>
>   Local time: Fri 2018-08-10 08:39:28 CEST
>
>   Universal time: Fri 2018-08-10 06:39:28 UTC
>
> RTC time: Fri 2018-08-10 06:39:28
>
>Time zone: Europe/Rome (CEST, +0200)
>
>System clock synchronized: yes
>
> systemd-timesyncd.service active: yes
>
>  RTC in local TZ: no
>
>
>
>
>
> What could I do?
>
> Thanks a lot
>
>
>
>
>
>
>
> *Da:* Karl Wright 
> *Inviato:* giovedì 9 agosto 2018 21:36
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Different time in Simple History Report
>
>
>
> Hi Mario.
>
> The pulldown allows you to select times based on the current (browser)
> time zone.
>
> The display is in *server* timezone.  That accounts for the difference.
>
>
>
> Karl
>
>
>
>
>
> On Thu, Aug 9, 2018 at 10:23 AM Bisonti Mario 
> wrote:
>
> Hallo
>
> I see a difference from the start time in “Simple History Report”
>
>
>
> It seems late of 2 hours.
>
>
>
> Have I to set timezone for this report?
>
>
>
> Thanks a lot
>
> See the attachment
>
>
>
>
>
>

R: Different time in Simple History Report

[Bisonti Mario]

Hallo Karl.
My server timezone is set as the browser timezone (europe/Rome) as you can see, 
but the list is two hour less my time zone.
So, it seems that the list uses the “universal time” instead of time zone

administrator@sengvivv01:~$ timedatectl
  Local time: Fri 2018-08-10 08:39:28 CEST
  Universal time: Fri 2018-08-10 06:39:28 UTC
RTC time: Fri 2018-08-10 06:39:28
   Time zone: Europe/Rome (CEST, +0200)
   System clock synchronized: yes
systemd-timesyncd.service active: yes
 RTC in local TZ: no


What could I do?
Thanks a lot



Da: Karl Wright 
Inviato: giovedì 9 agosto 2018 21:36
A: user@manifoldcf.apache.org
Oggetto: Re: Different time in Simple History Report

Hi Mario.

The pulldown allows you to select times based on the current (browser) time 
zone.

The display is in *server* timezone.  That accounts for the difference.

Karl


On Thu, Aug 9, 2018 at 10:23 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
Hallo
I see a difference from the start time in “Simple History Report”

It seems late of 2 hours.

Have I to set timezone for this report?

Thanks a lot
See the attachment


[cid:image003.jpg@01D42FFB.CFEC1780]