[GUMP@vmgump-vm3]: Project tomcat-trunk-test-nio2 (in module tomcat-trunk) failed

2018-08-17 Thread Bill Barker
To whom it may engage...

This is an automated request, but not an unsolicited one. For 
more information please visit http://gump.apache.org/nagged.html, 
and/or contact the folk at gene...@gump.apache.org.

Project tomcat-trunk-test-nio2 has an issue affecting its community integration.
This issue affects 1 projects,
 and has been outstanding for 4 runs.
The current state of this project is 'Failed', with reason 'Build Failed'.
For reference only, the following projects are affected by this:
- tomcat-trunk-test-nio2 :  Tomcat 9.x, a web server implementing the Java 
Servlet 4.0,
...


Full details are available at:
http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-nio2/index.html

That said, some information snippets are provided here.

The following annotations (debug/informational/warning/error messages) were 
provided:
 -DEBUG- Dependency on bnd exists, no need to add for property bndlib.jar.
 -INFO- Failed with reason build failed
 -INFO- Project Reports in: 
/srv/gump/public/workspace/tomcat-trunk/output/logs-NIO2
 -INFO- Project Reports in: 
/srv/gump/public/workspace/tomcat-trunk/output/test-tmp-NIO2/logs
 -WARNING- No directory 
[/srv/gump/public/workspace/tomcat-trunk/output/test-tmp-NIO2/logs]



The following work was performed:
http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-nio2/gump_work/build_tomcat-trunk_tomcat-trunk-test-nio2.html
Work Name: build_tomcat-trunk_tomcat-trunk-test-nio2 (Type: Build)
Work ended in a state of : Failed
Elapsed: 27 mins 11 secs
Command Line: /usr/lib/jvm/java-8-oracle/bin/java -Djava.awt.headless=true 
-Dbuild.sysclasspath=only -Dsun.zip.disableMemoryMapping=true 
org.apache.tools.ant.Main -Dgump.merge=/srv/gump/public/gump/work/merge.xml 
-Djunit.jar=/srv/gump/public/workspace/junit/target/junit-4.13-SNAPSHOT.jar 
-Djava.net.preferIPv4Stack=/srv/gump/public/workspace/tomcat-trunk/true 
-Dobjenesis.jar=/srv/gump/public/workspace/objenesis/main/target/objenesis-2.7-SNAPSHOT.jar
 -Dtest.reports=output/logs-NIO2 -Dexecute.test.nio2=true 
-Dexamples.sources.skip=true 
-Dbase.path=/srv/gump/public/workspace/tomcat-trunk/tomcat-build-libs 
-Djdt.jar=/srv/gump/packages/eclipse/plugins/R-4.7.3a-201803300640/ecj-4.7.3a.jar
 -Dbndlib.jar=/srv/gump/packages/bnd/bndlib-4.0.0/biz.aQute.bndlib-4.0.0.jar 
-Dcommons-daemon.jar=/srv/gump/public/workspace/apache-commons/daemon/target/commons-daemon-1.1.1-SNAPSHOT.jar
 
-Dtest.openssl.path=/srv/gump/public/workspace/openssl-master/dest-20180818/bin/openssl
 -Dtest.temp=output/test-tmp-NIO2
  -Dtest.accesslog=true -Dexecute.test.nio=false 
-Dbnd.jar=/srv/gump/packages/bnd/bnd-4.0.0/biz.aQute.bnd-4.0.0.jar 
-Dexecute.test.apr=false -Dtest.excludePerformance=true -Dtest.relaxTiming=true 
-Deasymock.jar=/srv/gump/public/workspace/easymock/core/target/easymock-3.7-SNAPSHOT.jar
 -Dhamcrest.jar=/srv/gump/packages/hamcrest/hamcrest-core-1.3.jar 
-Dcglib.jar=/srv/gump/packages/cglib/cglib-nodep-2.2.jar test 
[Working Directory: /srv/gump/public/workspace/tomcat-trunk]
CLASSPATH: 
/usr/lib/jvm/java-8-oracle/lib/tools.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/webapps/examples/WEB-INF/classes:/srv/gump/public/workspace/tomcat-trunk/output/testclasses:/srv/gump/public/workspace/ant/dist/lib/ant.jar:/srv/gump/public/workspace/ant/dist/lib/ant-launcher.jar:/srv/gump/public/workspace/ant/dist/lib/ant-jmf.jar:/srv/gump/public/workspace/ant/dist/lib/ant-junit.jar:/srv/gump/public/workspace/ant/dist/lib/ant-junit4.jar:/srv/gump/public/workspace/ant/dist/lib/ant-swing.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-resolver.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-xalan2.jar:/srv/gump/public/workspace/xml-commons/java/build/resolver.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/bin/bootstrap.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/bin/tomcat-juli.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/annotations-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/servlet-api.ja
 

[Bug 62635] New: Async servlet over HTTP/2 response.flushBuffer() intermittently fails

2018-08-17 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=62635

Bug ID: 62635
   Summary: Async servlet over HTTP/2 response.flushBuffer()
intermittently fails
   Product: Tomcat 9
   Version: 9.0.x
  Hardware: PC
OS: Linux
Status: NEW
  Severity: normal
  Priority: P2
 Component: Servlet
  Assignee: dev@tomcat.apache.org
  Reporter: zdap...@google.com
  Target Milestone: -

Sorry to bug you again. Now we were almost there, just had some failures
occasionally.

Test case:

import java.io.IOException;
import javax.servlet.AsyncContext;
import javax.servlet.ServletOutputStream;
import javax.servlet.WriteListener;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

@WebServlet(urlPatterns = {"/asyncflush"}, asyncSupported = true)
public class AsyncFlush extends HttpServlet {
  @Override
  protected void doGet(HttpServletRequest request, HttpServletResponse
response)
  throws IOException {
final AsyncContext asyncContext = request.startAsync();

response.setStatus(HttpServletResponse.SC_OK);
response.setContentType("application/binary");

final ServletOutputStream output = response.getOutputStream();
output.setWriteListener(new WriteListener() {
  int j;
  byte[] bytes = new byte[1000];
  int expectedBytesSentOut;

  @Override
  public void onWritePossible() throws IOException {
System.out.println("onWritePossible called");
if (j > 2000) {
  System.out.println("complete");
  System.out.println("expected bytes sent out: " +
expectedBytesSentOut);
  asyncContext.complete();
  return;
}

while(output.isReady()) {
  if (j % 5 == 4) {
System.out.println("start flush, j = " + j);
response.flushBuffer();
System.out.println("end flush, j = " + j);
  } else {
System.out.println("start write, j = " + j);
output.write(bytes);
expectedBytesSentOut += 1000;
System.out.println("end write, j = " + j);
  }
  j++;
}

System.out.println("output.isReady() = " + false);
  }

  @Override
  public void onError(Throwable t) {
t.printStackTrace();
  }
});
  }
}


Run the client command:
$ nghttp "http://0.0.0.0:8080/asyncflush; -v


The flush operation hangs up.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 62628] Async servlet over HTTP/2 non-blocking write does not work if client sets custom SETTINGS_INITIAL_WINDOW_SIZE

2018-08-17 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=62628

--- Comment #5 from Dapeng Zhang  ---
Thanks a lot for the fix! Now almost all my integrated tests passed.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



buildbot success in on tomcat-trunk

2018-08-17 Thread buildbot
The Buildbot has detected a restored build on builder tomcat-trunk while 
building . Full details are available at:
https://ci.apache.org/builders/tomcat-trunk/builds/3530

Buildbot URL: https://ci.apache.org/

Buildslave for this Build: silvanus_ubuntu

Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-commit' 
triggered this build
Build Source Stamp: [branch tomcat/trunk] 1838286
Blamelist: markt

Build succeeded!

Sincerely,
 -The Buildbot




-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



buildbot failure in on tomcat-trunk

2018-08-17 Thread buildbot
The Buildbot has detected a new failure on builder tomcat-trunk while building 
. Full details are available at:
https://ci.apache.org/builders/tomcat-trunk/builds/3529

Buildbot URL: https://ci.apache.org/

Buildslave for this Build: silvanus_ubuntu

Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-commit' 
triggered this build
Build Source Stamp: [branch tomcat/trunk] 1838281
Blamelist: markt

BUILD FAILED: failed compile_1

Sincerely,
 -The Buildbot




-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



svn commit: r28814 - /dev/tomcat/tomcat-8/v8.5.33/ /release/tomcat/tomcat-8/v8.5.33/

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 20:51:16 2018
New Revision: 28814

Log:
Release Apache Tomcat 8.5.33

Added:
release/tomcat/tomcat-8/v8.5.33/
  - copied from r28813, dev/tomcat/tomcat-8/v8.5.33/
Removed:
dev/tomcat/tomcat-8/v8.5.33/


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



svn commit: r1838288 - /tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 20:49:48 2018
New Revision: 1838288

URL: http://svn.apache.org/viewvc?rev=1838288=rev
Log:
Update release date

Modified:
tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml

Modified: tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml
URL: 
http://svn.apache.org/viewvc/tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml?rev=1838288=1838287=1838288=diff
==
--- tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml (original)
+++ tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml Fri Aug 17 20:49:48 2018
@@ -80,7 +80,7 @@
 
   
 
-
+
   
 
   



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[VOTE][RESULT] Release Apache Tomcat 8.5.33

2018-08-17 Thread Mark Thomas
The following votes were cast:

Binding:
+1: markt, fschumacher, violetagg

No other votes were cast.

The vote therefore passes.

Thanks to everyone who contributed towards this release.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 62628] Async servlet over HTTP/2 non-blocking write does not work if client sets custom SETTINGS_INITIAL_WINDOW_SIZE

2018-08-17 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=62628

Mark Thomas  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution|--- |FIXED

--- Comment #4 from Mark Thomas  ---
Fixed in:
- trunk for 9.0.12 onwards
- 8.5.x for 8.5.34 onwards

Thanks again for the bug report.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



svn commit: r1838287 - in /tomcat/tc8.5.x/trunk: ./ test/org/apache/coyote/http2/TestAsync.java webapps/docs/changelog.xml

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 20:46:21 2018
New Revision: 1838287

URL: http://svn.apache.org/viewvc?rev=1838287=rev
Log:
Expand Async + HTTP/2 tests to cover larger window sizes. Confirm previous fix 
fixes BZ 62628.

Modified:
tomcat/tc8.5.x/trunk/   (props changed)
tomcat/tc8.5.x/trunk/test/org/apache/coyote/http2/TestAsync.java
tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml

Propchange: tomcat/tc8.5.x/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Aug 17 20:46:21 2018
@@ -1,2 +1,2 @@
 /tomcat/tc8.0.x/trunk:1809644
-/tomcat/trunk:1734785,1734799,1734845,1734928,1735041,1735044,1735480,1735577,1735597,1735599-1735600,1735615,1736145,1736162,1736209,1736280,1736297,1736299,1736489,1736646,1736703,1736836,1736849,1737104-1737105,1737112,1737117,1737119-1737120,1737155,1737157,1737192,1737280,1737339,1737632,1737664,1737715,1737748,1737785,1737834,1737860,1737903,1737959,1738005,1738007,1738014-1738015,1738018,1738022,1738039,1738043,1738059-1738060,1738147,1738149,1738174-1738175,1738261,1738589,1738623-1738625,1738643,1738816,1738850,1738855,1738946-1738948,1738953-1738954,1738979,1738982,1739079-1739081,1739087,1739113,1739153,1739172,1739176,1739191,1739474,1739492,1739726,1739762,1739775,1739814,1739817-1739818,1739975,1740131,1740324,1740465,1740495,1740508-1740509,1740520,1740535,1740707,1740803,1740810,1740969,1740980,1740991,1740997,1741015,1741033,1741036,1741058,1741060,1741080,1741147,1741159,1741164,1741173,1741181,1741190,1741197,1741202,1741208,1741213,1741221,1741225,1741232,1741409
 
,1741501,1741677,1741892,1741896,1741984,1742023,1742042,1742071,1742090,1742093,1742101,1742105,1742111,1742139,1742146,1742148,1742166,1742181,1742184,1742187,1742246,1742248-1742251,1742263-1742264,1742268,1742276,1742369,1742387,1742448,1742509-1742512,1742917,1742919,1742933,1742975-1742976,1742984,1742986,1743019,1743115,1743117,1743124-1743125,1743134,1743425,1743554,1743679,1743696-1743698,1743700-1743701,1744058,1744064-1744065,1744125,1744149,1744194,1744229,1744270,1744323,1744432,1744684,1744697,1744705,1744713,1744760,1744786,1745083,1745142-1745143,1745145,1745177,1745179-1745180,1745227,1745248,1745254,1745337,1745467,1745473,1745535,1745576,1745735,1745744,1746304,1746306-1746307,1746319,1746327,1746338,1746340-1746341,1746344,1746427,1746441,1746473,1746490,1746492,1746495-1746496,1746499-1746501,1746503-1746507,1746509,1746549,1746551,1746554,1746556,1746558,1746584,1746620,1746649,1746724,1746939,1746989,1747014,1747028,1747035,1747210,1747225,1747234,1747253,1747
 
404,1747506,1747536,1747924,1747980,1747993,1748001,1748253,1748452,1748547,1748629,1748676,1748715,1749287,1749296,1749328,1749373,1749465,1749506,1749508,1749665-1749666,1749763,1749865-1749866,1749898,1749978,1749980,1750011,1750015,1750056,1750480,1750617,1750634,1750692,1750697,1750700,1750703,1750707,1750714,1750718,1750723,1750774,1750899,1750975,1750995,1751061,1751097,1751173,1751438,1751447,1751463,1751702,1752212,1752737,1752745,1753078,1753080,1753358,1753363,1754111,1754140-1754141,1754281,1754310,1754445,1754467,1754494,1754496,1754528,1754532-1754533,1754613,1754714,1754874,1754941,1754944,1754950-1754951,1755005,1755007,1755009,1755132,1755180-1755181,1755185,1755190,1755204-1755206,1755208,1755214,1755224,1755227,1755230,1755629,1755646-1755647,1755650,1755653,1755675,1755680,1755683,1755693,1755717,1755731-1755737,1755812,1755828,1755884,1755890,1755918-1755919,1755942,1755958,1755960,1755970,1755993,1756013,1756019,1756039,1756056,1756083-1756114,1756175,1756288-1
 
756289,1756408-1756410,1756778,1756798,1756878,1756898,1756939,1757123-1757124,1757126,1757128,1757132-1757133,1757136,1757145,1757167-1757168,1757175,1757180,1757182,1757195,1757271,1757278,1757347,1757353-1757354,1757363,1757374,1757399,1757406,1757408,1757485,1757495,1757499,1757527,1757578,1757684,1757722,1757727,1757790,1757799,1757813,1757853,1757883,1757903,1757976,1757997,1758000,1758058,1758072-1758075,1758078-1758079,1758223,1758257,1758261,1758276,1758292,1758369,1758378-1758383,1758421,1758423,1758425-1758427,1758430,1758443,1758448,1758459,1758483,1758486-1758487,1758499,1758525,1758556,1758580,1758582,1758584,1758588,1758842,1759019,1759212,1759224,1759227,1759252,1759274,1759513-1759516,1759611,1759757,1759785-1759790,1760005,1760022,1760109-1760110,1760135,1760200-1760201,1760227,1760300,1760397,1760446,1760454,1760640,1760648,1761057,1761422,1761491,1761498,1761500-1761501,1761550,1761553,1761572,1761574,1761625-1761626,1761628,1761682,1761740,1761752,1762051-176205
 

svn commit: r1838286 - in /tomcat/trunk: test/org/apache/coyote/http2/TestAsync.java webapps/docs/changelog.xml

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 20:45:42 2018
New Revision: 1838286

URL: http://svn.apache.org/viewvc?rev=1838286=rev
Log:
Expand Async + HTTP/2 tests to cover larger window sizes. Confirm previous fix 
fixes BZ 62628.

Modified:
tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java
tomcat/trunk/webapps/docs/changelog.xml

Modified: tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java
URL: 
http://svn.apache.org/viewvc/tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java?rev=1838286=1838285=1838286=diff
==
--- tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java (original)
+++ tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java Fri Aug 17 
20:45:42 2018
@@ -46,6 +46,7 @@ import org.apache.catalina.startup.Tomca
  * Based on
  * https://bz.apache.org/bugzilla/show_bug.cgi?id=62614
  * https://bz.apache.org/bugzilla/show_bug.cgi?id=62620
+ * https://bz.apache.org/bugzilla/show_bug.cgi?id=62628
  */
 @RunWith(Parameterized.class)
 public class TestAsync extends Http2TestBase {
@@ -53,7 +54,8 @@ public class TestAsync extends Http2Test
 private static final int BLOCK_SIZE = 0x8000;
 
 @Parameterized.Parameters(name = "{index}: expandConnectionFirst[{0}], " +
-"connectionUnlimited[{1}], streamUnlimited[{2}], 
useNonContainerThreadForWrite[{3}]")
+"connectionUnlimited[{1}], streamUnlimited[{2}], 
useNonContainerThreadForWrite[{3}]," +
+"largeInitialWindow[{4}]")
 public static Collection parameters() {
 Boolean[] booleans = new Boolean[] { Boolean.FALSE, Boolean.TRUE };
 List parameterSets = new ArrayList<>();
@@ -62,10 +64,12 @@ public class TestAsync extends Http2Test
 for (Boolean connectionUnlimited : booleans) {
 for (Boolean streamUnlimited : booleans) {
 for (Boolean useNonContainerThreadForWrite : booleans) {
-parameterSets.add(new Object[] {
-expandConnectionFirst, connectionUnlimited, 
streamUnlimited,
-useNonContainerThreadForWrite
-});
+for (Boolean largeInitialWindow : booleans) {
+parameterSets.add(new Object[] {
+expandConnectionFirst, 
connectionUnlimited, streamUnlimited,
+useNonContainerThreadForWrite, 
largeInitialWindow
+});
+}
 }
 }
 }
@@ -78,20 +82,23 @@ public class TestAsync extends Http2Test
 private final boolean connectionUnlimited;
 private final boolean streamUnlimited;
 private final boolean useNonContainerThreadForWrite;
+private final boolean largeInitialWindow;
 
 
 public TestAsync(boolean expandConnectionFirst, boolean 
connectionUnlimited,
-boolean streamUnlimited, boolean useNonContainerThreadForWrite) {
+boolean streamUnlimited, boolean useNonContainerThreadForWrite,
+boolean largeInitialWindow) {
 this.expandConnectionFirst = expandConnectionFirst;
 this.connectionUnlimited = connectionUnlimited;
 this.streamUnlimited = streamUnlimited;
 this.useNonContainerThreadForWrite = useNonContainerThreadForWrite;
+this.largeInitialWindow = largeInitialWindow;
 }
 
 
 @Test
 public void testEmptyWindow() throws Exception {
-int blockCount = 4;
+int blockCount = 8;
 
 enableHttp2();
 
@@ -106,19 +113,32 @@ public class TestAsync extends Http2Test
 ctxt.addServletMappingDecoded("/async", "async");
 tomcat.start();
 
+int startingWindowSize;
+
 openClientConnection();
 doHttpUpgrade();
 sendClientPreface();
 validateHttp2InitialResponse();
 
+// Reset connection window size after intial response
+sendWindowUpdate(0, SimpleServlet.CONTENT_LENGTH);
+
+if (largeInitialWindow) {
+startingWindowSize = ((1 << 17) - 1);
+SettingValue sv =
+new SettingValue(Setting.INITIAL_WINDOW_SIZE.getId(), 
startingWindowSize);
+sendSettings(0, false, sv);
+// Test code assumes connection window and stream window size are 
the same at the start
+sendWindowUpdate(0, startingWindowSize - 
ConnectionSettingsBase.DEFAULT_INITIAL_WINDOW_SIZE);
+} else {
+startingWindowSize = 
ConnectionSettingsBase.DEFAULT_INITIAL_WINDOW_SIZE;
+}
+
 byte[] frameHeader = new byte[9];
 ByteBuffer headersPayload = ByteBuffer.allocate(128);
 buildGetRequest(frameHeader, headersPayload, null, 3, "/async");
 writeFrame(frameHeader, headersPayload);
 
-// Reset connection window size after intial response
-sendWindowUpdate(0, 

svn commit: r1838282 - in /tomcat/tc8.5.x/trunk: ./ test/org/apache/coyote/http2/TestAsync.java

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 20:07:38 2018
New Revision: 1838282

URL: http://svn.apache.org/viewvc?rev=1838282=rev
Log:
Expand async HTTP/2 tests to cover writes on non-container threads

Modified:
tomcat/tc8.5.x/trunk/   (props changed)
tomcat/tc8.5.x/trunk/test/org/apache/coyote/http2/TestAsync.java

Propchange: tomcat/tc8.5.x/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Aug 17 20:07:38 2018
@@ -1,2 +1,2 @@
 /tomcat/tc8.0.x/trunk:1809644
-/tomcat/trunk:1734785,1734799,1734845,1734928,1735041,1735044,1735480,1735577,1735597,1735599-1735600,1735615,1736145,1736162,1736209,1736280,1736297,1736299,1736489,1736646,1736703,1736836,1736849,1737104-1737105,1737112,1737117,1737119-1737120,1737155,1737157,1737192,1737280,1737339,1737632,1737664,1737715,1737748,1737785,1737834,1737860,1737903,1737959,1738005,1738007,1738014-1738015,1738018,1738022,1738039,1738043,1738059-1738060,1738147,1738149,1738174-1738175,1738261,1738589,1738623-1738625,1738643,1738816,1738850,1738855,1738946-1738948,1738953-1738954,1738979,1738982,1739079-1739081,1739087,1739113,1739153,1739172,1739176,1739191,1739474,1739492,1739726,1739762,1739775,1739814,1739817-1739818,1739975,1740131,1740324,1740465,1740495,1740508-1740509,1740520,1740535,1740707,1740803,1740810,1740969,1740980,1740991,1740997,1741015,1741033,1741036,1741058,1741060,1741080,1741147,1741159,1741164,1741173,1741181,1741190,1741197,1741202,1741208,1741213,1741221,1741225,1741232,1741409
 
,1741501,1741677,1741892,1741896,1741984,1742023,1742042,1742071,1742090,1742093,1742101,1742105,1742111,1742139,1742146,1742148,1742166,1742181,1742184,1742187,1742246,1742248-1742251,1742263-1742264,1742268,1742276,1742369,1742387,1742448,1742509-1742512,1742917,1742919,1742933,1742975-1742976,1742984,1742986,1743019,1743115,1743117,1743124-1743125,1743134,1743425,1743554,1743679,1743696-1743698,1743700-1743701,1744058,1744064-1744065,1744125,1744149,1744194,1744229,1744270,1744323,1744432,1744684,1744697,1744705,1744713,1744760,1744786,1745083,1745142-1745143,1745145,1745177,1745179-1745180,1745227,1745248,1745254,1745337,1745467,1745473,1745535,1745576,1745735,1745744,1746304,1746306-1746307,1746319,1746327,1746338,1746340-1746341,1746344,1746427,1746441,1746473,1746490,1746492,1746495-1746496,1746499-1746501,1746503-1746507,1746509,1746549,1746551,1746554,1746556,1746558,1746584,1746620,1746649,1746724,1746939,1746989,1747014,1747028,1747035,1747210,1747225,1747234,1747253,1747
 
404,1747506,1747536,1747924,1747980,1747993,1748001,1748253,1748452,1748547,1748629,1748676,1748715,1749287,1749296,1749328,1749373,1749465,1749506,1749508,1749665-1749666,1749763,1749865-1749866,1749898,1749978,1749980,1750011,1750015,1750056,1750480,1750617,1750634,1750692,1750697,1750700,1750703,1750707,1750714,1750718,1750723,1750774,1750899,1750975,1750995,1751061,1751097,1751173,1751438,1751447,1751463,1751702,1752212,1752737,1752745,1753078,1753080,1753358,1753363,1754111,1754140-1754141,1754281,1754310,1754445,1754467,1754494,1754496,1754528,1754532-1754533,1754613,1754714,1754874,1754941,1754944,1754950-1754951,1755005,1755007,1755009,1755132,1755180-1755181,1755185,1755190,1755204-1755206,1755208,1755214,1755224,1755227,1755230,1755629,1755646-1755647,1755650,1755653,1755675,1755680,1755683,1755693,1755717,1755731-1755737,1755812,1755828,1755884,1755890,1755918-1755919,1755942,1755958,1755960,1755970,1755993,1756013,1756019,1756039,1756056,1756083-1756114,1756175,1756288-1
 
756289,1756408-1756410,1756778,1756798,1756878,1756898,1756939,1757123-1757124,1757126,1757128,1757132-1757133,1757136,1757145,1757167-1757168,1757175,1757180,1757182,1757195,1757271,1757278,1757347,1757353-1757354,1757363,1757374,1757399,1757406,1757408,1757485,1757495,1757499,1757527,1757578,1757684,1757722,1757727,1757790,1757799,1757813,1757853,1757883,1757903,1757976,1757997,1758000,1758058,1758072-1758075,1758078-1758079,1758223,1758257,1758261,1758276,1758292,1758369,1758378-1758383,1758421,1758423,1758425-1758427,1758430,1758443,1758448,1758459,1758483,1758486-1758487,1758499,1758525,1758556,1758580,1758582,1758584,1758588,1758842,1759019,1759212,1759224,1759227,1759252,1759274,1759513-1759516,1759611,1759757,1759785-1759790,1760005,1760022,1760109-1760110,1760135,1760200-1760201,1760227,1760300,1760397,1760446,1760454,1760640,1760648,1761057,1761422,1761491,1761498,1761500-1761501,1761550,1761553,1761572,1761574,1761625-1761626,1761628,1761682,1761740,1761752,1762051-176205
 

svn commit: r1838281 - /tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 20:07:19 2018
New Revision: 1838281

URL: http://svn.apache.org/viewvc?rev=1838281=rev
Log:
Expand async HTTP/2 tests to cover writes on non-container threads

Modified:
tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java

Modified: tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java
URL: 
http://svn.apache.org/viewvc/tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java?rev=1838281=1838280=1838281=diff
==
--- tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java (original)
+++ tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java Fri Aug 17 
20:07:19 2018
@@ -21,6 +21,10 @@ import java.nio.ByteBuffer;
 import java.util.ArrayList;
 import java.util.Collection;
 import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
 
 import javax.servlet.AsyncContext;
 import javax.servlet.ServletOutputStream;
@@ -49,7 +53,7 @@ public class TestAsync extends Http2Test
 private static final int BLOCK_SIZE = 0x8000;
 
 @Parameterized.Parameters(name = "{index}: expandConnectionFirst[{0}], " +
-"connectionUnlimited[{1}], streamUnlimited[{2}]")
+"connectionUnlimited[{1}], streamUnlimited[{2}], 
useNonContainerThreadForWrite[{3}]")
 public static Collection parameters() {
 Boolean[] booleans = new Boolean[] { Boolean.FALSE, Boolean.TRUE };
 List parameterSets = new ArrayList<>();
@@ -57,9 +61,12 @@ public class TestAsync extends Http2Test
 for (Boolean expandConnectionFirst : booleans) {
 for (Boolean connectionUnlimited : booleans) {
 for (Boolean streamUnlimited : booleans) {
-parameterSets.add(new Object[] {
-expandConnectionFirst, connectionUnlimited, 
streamUnlimited
-});
+for (Boolean useNonContainerThreadForWrite : booleans) {
+parameterSets.add(new Object[] {
+expandConnectionFirst, connectionUnlimited, 
streamUnlimited,
+useNonContainerThreadForWrite
+});
+}
 }
 }
 }
@@ -70,13 +77,15 @@ public class TestAsync extends Http2Test
 private final boolean expandConnectionFirst;
 private final boolean connectionUnlimited;
 private final boolean streamUnlimited;
+private final boolean useNonContainerThreadForWrite;
 
 
 public TestAsync(boolean expandConnectionFirst, boolean 
connectionUnlimited,
-boolean streamUnlimited) {
+boolean streamUnlimited, boolean useNonContainerThreadForWrite) {
 this.expandConnectionFirst = expandConnectionFirst;
 this.connectionUnlimited = connectionUnlimited;
 this.streamUnlimited = streamUnlimited;
+this.useNonContainerThreadForWrite = useNonContainerThreadForWrite;
 }
 
 
@@ -91,7 +100,8 @@ public class TestAsync extends Http2Test
 Context ctxt = tomcat.addContext("", null);
 Tomcat.addServlet(ctxt, "simple", new SimpleServlet());
 ctxt.addServletMappingDecoded("/simple", "simple");
-Wrapper w = Tomcat.addServlet(ctxt, "async", new 
AsyncServlet(blockCount));
+Wrapper w = Tomcat.addServlet(ctxt, "async",
+new AsyncServlet(blockCount, useNonContainerThreadForWrite));
 w.setAsyncSupported(true);
 ctxt.addServletMappingDecoded("/async", "async");
 tomcat.start();
@@ -173,11 +183,18 @@ public class TestAsync extends Http2Test
 private static final long serialVersionUID = 1L;
 
 private final int blockLimit;
+private final boolean useNonContainerThreadForWrite;
+private final ScheduledExecutorService scheduler = 
Executors.newScheduledThreadPool(1);
+private volatile Future future;
 
-public AsyncServlet(int blockLimit) {
+public AsyncServlet(int blockLimit, boolean 
useNonContainerThreadForWrite) {
 this.blockLimit = blockLimit;
+this.useNonContainerThreadForWrite = useNonContainerThreadForWrite;
 }
 
+/*
+ * Not thread-safe. OK for this test. NOt OK for use in the real world.
+ */
 @Override
 protected void doGet(HttpServletRequest request, HttpServletResponse 
response)
 throws IOException {
@@ -196,19 +213,41 @@ public class TestAsync extends Http2Test
 
 @Override
 public void onWritePossible() throws IOException {
+if (useNonContainerThreadForWrite) {
+future = scheduler.schedule(new Runnable() {
+
+@Override
+public void run() {
+

svn commit: r1838280 - in /tomcat/tc8.5.x/trunk: ./ conf/logging.properties

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 19:53:19 2018
New Revision: 1838280

URL: http://svn.apache.org/viewvc?rev=1838280=rev
Log:
Undo debugging change

Modified:
tomcat/tc8.5.x/trunk/   (props changed)
tomcat/tc8.5.x/trunk/conf/logging.properties

Propchange: tomcat/tc8.5.x/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Aug 17 19:53:19 2018
@@ -1,2 +1,2 @@
 /tomcat/tc8.0.x/trunk:1809644
-/tomcat/trunk:1734785,1734799,1734845,1734928,1735041,1735044,1735480,1735577,1735597,1735599-1735600,1735615,1736145,1736162,1736209,1736280,1736297,1736299,1736489,1736646,1736703,1736836,1736849,1737104-1737105,1737112,1737117,1737119-1737120,1737155,1737157,1737192,1737280,1737339,1737632,1737664,1737715,1737748,1737785,1737834,1737860,1737903,1737959,1738005,1738007,1738014-1738015,1738018,1738022,1738039,1738043,1738059-1738060,1738147,1738149,1738174-1738175,1738261,1738589,1738623-1738625,1738643,1738816,1738850,1738855,1738946-1738948,1738953-1738954,1738979,1738982,1739079-1739081,1739087,1739113,1739153,1739172,1739176,1739191,1739474,1739492,1739726,1739762,1739775,1739814,1739817-1739818,1739975,1740131,1740324,1740465,1740495,1740508-1740509,1740520,1740535,1740707,1740803,1740810,1740969,1740980,1740991,1740997,1741015,1741033,1741036,1741058,1741060,1741080,1741147,1741159,1741164,1741173,1741181,1741190,1741197,1741202,1741208,1741213,1741221,1741225,1741232,1741409
 
,1741501,1741677,1741892,1741896,1741984,1742023,1742042,1742071,1742090,1742093,1742101,1742105,1742111,1742139,1742146,1742148,1742166,1742181,1742184,1742187,1742246,1742248-1742251,1742263-1742264,1742268,1742276,1742369,1742387,1742448,1742509-1742512,1742917,1742919,1742933,1742975-1742976,1742984,1742986,1743019,1743115,1743117,1743124-1743125,1743134,1743425,1743554,1743679,1743696-1743698,1743700-1743701,1744058,1744064-1744065,1744125,1744149,1744194,1744229,1744270,1744323,1744432,1744684,1744697,1744705,1744713,1744760,1744786,1745083,1745142-1745143,1745145,1745177,1745179-1745180,1745227,1745248,1745254,1745337,1745467,1745473,1745535,1745576,1745735,1745744,1746304,1746306-1746307,1746319,1746327,1746338,1746340-1746341,1746344,1746427,1746441,1746473,1746490,1746492,1746495-1746496,1746499-1746501,1746503-1746507,1746509,1746549,1746551,1746554,1746556,1746558,1746584,1746620,1746649,1746724,1746939,1746989,1747014,1747028,1747035,1747210,1747225,1747234,1747253,1747
 
404,1747506,1747536,1747924,1747980,1747993,1748001,1748253,1748452,1748547,1748629,1748676,1748715,1749287,1749296,1749328,1749373,1749465,1749506,1749508,1749665-1749666,1749763,1749865-1749866,1749898,1749978,1749980,1750011,1750015,1750056,1750480,1750617,1750634,1750692,1750697,1750700,1750703,1750707,1750714,1750718,1750723,1750774,1750899,1750975,1750995,1751061,1751097,1751173,1751438,1751447,1751463,1751702,1752212,1752737,1752745,1753078,1753080,1753358,1753363,1754111,1754140-1754141,1754281,1754310,1754445,1754467,1754494,1754496,1754528,1754532-1754533,1754613,1754714,1754874,1754941,1754944,1754950-1754951,1755005,1755007,1755009,1755132,1755180-1755181,1755185,1755190,1755204-1755206,1755208,1755214,1755224,1755227,1755230,1755629,1755646-1755647,1755650,1755653,1755675,1755680,1755683,1755693,1755717,1755731-1755737,1755812,1755828,1755884,1755890,1755918-1755919,1755942,1755958,1755960,1755970,1755993,1756013,1756019,1756039,1756056,1756083-1756114,1756175,1756288-1
 
756289,1756408-1756410,1756778,1756798,1756878,1756898,1756939,1757123-1757124,1757126,1757128,1757132-1757133,1757136,1757145,1757167-1757168,1757175,1757180,1757182,1757195,1757271,1757278,1757347,1757353-1757354,1757363,1757374,1757399,1757406,1757408,1757485,1757495,1757499,1757527,1757578,1757684,1757722,1757727,1757790,1757799,1757813,1757853,1757883,1757903,1757976,1757997,1758000,1758058,1758072-1758075,1758078-1758079,1758223,1758257,1758261,1758276,1758292,1758369,1758378-1758383,1758421,1758423,1758425-1758427,1758430,1758443,1758448,1758459,1758483,1758486-1758487,1758499,1758525,1758556,1758580,1758582,1758584,1758588,1758842,1759019,1759212,1759224,1759227,1759252,1759274,1759513-1759516,1759611,1759757,1759785-1759790,1760005,1760022,1760109-1760110,1760135,1760200-1760201,1760227,1760300,1760397,1760446,1760454,1760640,1760648,1761057,1761422,1761491,1761498,1761500-1761501,1761550,1761553,1761572,1761574,1761625-1761626,1761628,1761682,1761740,1761752,1762051-176205
 

svn commit: r1838279 - /tomcat/trunk/conf/logging.properties

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 19:53:01 2018
New Revision: 1838279

URL: http://svn.apache.org/viewvc?rev=1838279=rev
Log:
Undo debugging change

Modified:
tomcat/trunk/conf/logging.properties

Modified: tomcat/trunk/conf/logging.properties
URL: 
http://svn.apache.org/viewvc/tomcat/trunk/conf/logging.properties?rev=1838279=1838278=1838279=diff
==
--- tomcat/trunk/conf/logging.properties (original)
+++ tomcat/trunk/conf/logging.properties Fri Aug 17 19:53:01 2018
@@ -68,7 +68,7 @@ org.apache.catalina.core.ContainerBase.[
 #org.apache.jasper.compiler.TldLocationsCache.level = FINE
 
 # To see debug messages for HTTP/2 handling, uncomment the following line:
-org.apache.coyote.http2.level = FINE
+#org.apache.coyote.http2.level = FINE
 
 # To see debug messages for WebSocket handling, uncomment the following line:
 #org.apache.tomcat.websocket.level = FINE



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



svn commit: r1838278 - in /tomcat/tc8.5.x/trunk: ./ conf/logging.properties test/org/apache/coyote/http2/TestAsync.java

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 19:52:29 2018
New Revision: 1838278

URL: http://svn.apache.org/viewvc?rev=1838278=rev
Log:
Refactor to a parameterized test

Modified:
tomcat/tc8.5.x/trunk/   (props changed)
tomcat/tc8.5.x/trunk/conf/logging.properties
tomcat/tc8.5.x/trunk/test/org/apache/coyote/http2/TestAsync.java

Propchange: tomcat/tc8.5.x/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Aug 17 19:52:29 2018
@@ -1,2 +1,2 @@
 /tomcat/tc8.0.x/trunk:1809644
-/tomcat/trunk:1734785,1734799,1734845,1734928,1735041,1735044,1735480,1735577,1735597,1735599-1735600,1735615,1736145,1736162,1736209,1736280,1736297,1736299,1736489,1736646,1736703,1736836,1736849,1737104-1737105,1737112,1737117,1737119-1737120,1737155,1737157,1737192,1737280,1737339,1737632,1737664,1737715,1737748,1737785,1737834,1737860,1737903,1737959,1738005,1738007,1738014-1738015,1738018,1738022,1738039,1738043,1738059-1738060,1738147,1738149,1738174-1738175,1738261,1738589,1738623-1738625,1738643,1738816,1738850,1738855,1738946-1738948,1738953-1738954,1738979,1738982,1739079-1739081,1739087,1739113,1739153,1739172,1739176,1739191,1739474,1739492,1739726,1739762,1739775,1739814,1739817-1739818,1739975,1740131,1740324,1740465,1740495,1740508-1740509,1740520,1740535,1740707,1740803,1740810,1740969,1740980,1740991,1740997,1741015,1741033,1741036,1741058,1741060,1741080,1741147,1741159,1741164,1741173,1741181,1741190,1741197,1741202,1741208,1741213,1741221,1741225,1741232,1741409
 
,1741501,1741677,1741892,1741896,1741984,1742023,1742042,1742071,1742090,1742093,1742101,1742105,1742111,1742139,1742146,1742148,1742166,1742181,1742184,1742187,1742246,1742248-1742251,1742263-1742264,1742268,1742276,1742369,1742387,1742448,1742509-1742512,1742917,1742919,1742933,1742975-1742976,1742984,1742986,1743019,1743115,1743117,1743124-1743125,1743134,1743425,1743554,1743679,1743696-1743698,1743700-1743701,1744058,1744064-1744065,1744125,1744149,1744194,1744229,1744270,1744323,1744432,1744684,1744697,1744705,1744713,1744760,1744786,1745083,1745142-1745143,1745145,1745177,1745179-1745180,1745227,1745248,1745254,1745337,1745467,1745473,1745535,1745576,1745735,1745744,1746304,1746306-1746307,1746319,1746327,1746338,1746340-1746341,1746344,1746427,1746441,1746473,1746490,1746492,1746495-1746496,1746499-1746501,1746503-1746507,1746509,1746549,1746551,1746554,1746556,1746558,1746584,1746620,1746649,1746724,1746939,1746989,1747014,1747028,1747035,1747210,1747225,1747234,1747253,1747
 
404,1747506,1747536,1747924,1747980,1747993,1748001,1748253,1748452,1748547,1748629,1748676,1748715,1749287,1749296,1749328,1749373,1749465,1749506,1749508,1749665-1749666,1749763,1749865-1749866,1749898,1749978,1749980,1750011,1750015,1750056,1750480,1750617,1750634,1750692,1750697,1750700,1750703,1750707,1750714,1750718,1750723,1750774,1750899,1750975,1750995,1751061,1751097,1751173,1751438,1751447,1751463,1751702,1752212,1752737,1752745,1753078,1753080,1753358,1753363,1754111,1754140-1754141,1754281,1754310,1754445,1754467,1754494,1754496,1754528,1754532-1754533,1754613,1754714,1754874,1754941,1754944,1754950-1754951,1755005,1755007,1755009,1755132,1755180-1755181,1755185,1755190,1755204-1755206,1755208,1755214,1755224,1755227,1755230,1755629,1755646-1755647,1755650,1755653,1755675,1755680,1755683,1755693,1755717,1755731-1755737,1755812,1755828,1755884,1755890,1755918-1755919,1755942,1755958,1755960,1755970,1755993,1756013,1756019,1756039,1756056,1756083-1756114,1756175,1756288-1
 
756289,1756408-1756410,1756778,1756798,1756878,1756898,1756939,1757123-1757124,1757126,1757128,1757132-1757133,1757136,1757145,1757167-1757168,1757175,1757180,1757182,1757195,1757271,1757278,1757347,1757353-1757354,1757363,1757374,1757399,1757406,1757408,1757485,1757495,1757499,1757527,1757578,1757684,1757722,1757727,1757790,1757799,1757813,1757853,1757883,1757903,1757976,1757997,1758000,1758058,1758072-1758075,1758078-1758079,1758223,1758257,1758261,1758276,1758292,1758369,1758378-1758383,1758421,1758423,1758425-1758427,1758430,1758443,1758448,1758459,1758483,1758486-1758487,1758499,1758525,1758556,1758580,1758582,1758584,1758588,1758842,1759019,1759212,1759224,1759227,1759252,1759274,1759513-1759516,1759611,1759757,1759785-1759790,1760005,1760022,1760109-1760110,1760135,1760200-1760201,1760227,1760300,1760397,1760446,1760454,1760640,1760648,1761057,1761422,1761491,1761498,1761500-1761501,1761550,1761553,1761572,1761574,1761625-1761626,1761628,1761682,1761740,1761752,1762051-176205
 

svn commit: r1838277 - in /tomcat/trunk: conf/logging.properties test/org/apache/coyote/http2/TestAsync.java

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 19:51:44 2018
New Revision: 1838277

URL: http://svn.apache.org/viewvc?rev=1838277=rev
Log:
Refactor to a parameterized test

Modified:
tomcat/trunk/conf/logging.properties
tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java

Modified: tomcat/trunk/conf/logging.properties
URL: 
http://svn.apache.org/viewvc/tomcat/trunk/conf/logging.properties?rev=1838277=1838276=1838277=diff
==
--- tomcat/trunk/conf/logging.properties (original)
+++ tomcat/trunk/conf/logging.properties Fri Aug 17 19:51:44 2018
@@ -68,7 +68,7 @@ org.apache.catalina.core.ContainerBase.[
 #org.apache.jasper.compiler.TldLocationsCache.level = FINE
 
 # To see debug messages for HTTP/2 handling, uncomment the following line:
-#org.apache.coyote.http2.level = FINE
+org.apache.coyote.http2.level = FINE
 
 # To see debug messages for WebSocket handling, uncomment the following line:
 #org.apache.tomcat.websocket.level = FINE

Modified: tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java
URL: 
http://svn.apache.org/viewvc/tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java?rev=1838277=1838276=1838277=diff
==
--- tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java (original)
+++ tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java Fri Aug 17 
19:51:44 2018
@@ -18,6 +18,9 @@ package org.apache.coyote.http2;
 
 import java.io.IOException;
 import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
 
 import javax.servlet.AsyncContext;
 import javax.servlet.ServletOutputStream;
@@ -28,57 +31,57 @@ import javax.servlet.http.HttpServletRes
 
 import org.junit.Assert;
 import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
 
 import org.apache.catalina.Context;
 import org.apache.catalina.Wrapper;
 import org.apache.catalina.startup.Tomcat;
 
+/*
+ * Based on
+ * https://bz.apache.org/bugzilla/show_bug.cgi?id=62614
+ * https://bz.apache.org/bugzilla/show_bug.cgi?id=62620
+ */
+@RunWith(Parameterized.class)
 public class TestAsync extends Http2TestBase {
 
 private static final int BLOCK_SIZE = 0x8000;
 
-// https://bz.apache.org/bugzilla/show_bug.cgi?id=62614
-@Test
-public void testEmptyBothWindowsUpdateConnectionFirst() throws Exception {
-doEmptyWindowTest(true, false, false);
-}
-
-
-@Test
-public void testEmptyBothWindowsUpdateStreamFirst() throws Exception {
-doEmptyWindowTest(false, false, false);
-}
-
+@Parameterized.Parameters(name = "{index}: expandConnectionFirst[{0}], " +
+"connectionUnlimited[{1}], streamUnlimited[{2}]")
+public static Collection parameters() {
+Boolean[] booleans = new Boolean[] { Boolean.FALSE, Boolean.TRUE };
+List parameterSets = new ArrayList<>();
 
-@Test
-public void testEmptyConnectionWindowUpdateConnectionFirst() throws 
Exception {
-doEmptyWindowTest(true, false, true);
+for (Boolean expandConnectionFirst : booleans) {
+for (Boolean connectionUnlimited : booleans) {
+for (Boolean streamUnlimited : booleans) {
+parameterSets.add(new Object[] {
+expandConnectionFirst, connectionUnlimited, 
streamUnlimited
+});
+}
+}
+}
+return parameterSets;
 }
 
 
-@Test
-public void testEmptyConnectionWindowUpdateStreamFirst() throws Exception {
-doEmptyWindowTest(false, false, true);
-}
+private final boolean expandConnectionFirst;
+private final boolean connectionUnlimited;
+private final boolean streamUnlimited;
 
 
-@Test
-public void testEmptyStreamWindowUpdateConnectionFirst() throws Exception {
-doEmptyWindowTest(true, true, false);
+public TestAsync(boolean expandConnectionFirst, boolean 
connectionUnlimited,
+boolean streamUnlimited) {
+this.expandConnectionFirst = expandConnectionFirst;
+this.connectionUnlimited = connectionUnlimited;
+this.streamUnlimited = streamUnlimited;
 }
 
 
 @Test
-public void testEmptyStreamWindowUpdateStreamFirst() throws Exception {
-doEmptyWindowTest(false, true, false);
-}
-
-
-// No point testing when both Stream and Connection are unlimited
-
-
-private void doEmptyWindowTest(boolean expandConnectionFirst, boolean 
connectionUnlimited,
-boolean streamUnlimited) throws Exception {
+public void testEmptyWindow() throws Exception {
 int blockCount = 4;
 
 enableHttp2();
@@ -118,39 +121,42 @@ public class TestAsync extends Http2Test
 // Headers
 parser.readFrame(true);
 // Body
-int startingWindowSize = 
ConnectionSettingsBase.DEFAULT_INITIAL_WINDOW_SIZE;
-
- 

svn commit: r1838276 - in /tomcat/tc8.5.x/trunk: ./ java/org/apache/coyote/http2/ test/org/apache/coyote/http2/ webapps/docs/

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 19:26:24 2018
New Revision: 1838276

URL: http://svn.apache.org/viewvc?rev=1838276=rev
Log:
Additional fixes for output corruption of response bodies when writing large 
bodies using asynchronous processing over HTTP/2.

Added:
tomcat/tc8.5.x/trunk/test/org/apache/coyote/http2/TestAsync.java
  - copied unchanged from r1838275, 
tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java
Modified:
tomcat/tc8.5.x/trunk/   (props changed)
tomcat/tc8.5.x/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java
tomcat/tc8.5.x/trunk/java/org/apache/coyote/http2/Stream.java
tomcat/tc8.5.x/trunk/test/org/apache/coyote/http2/Http2TestBase.java
tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml

Propchange: tomcat/tc8.5.x/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Aug 17 19:26:24 2018
@@ -1,2 +1,2 @@
 /tomcat/tc8.0.x/trunk:1809644
-/tomcat/trunk:1734785,1734799,1734845,1734928,1735041,1735044,1735480,1735577,1735597,1735599-1735600,1735615,1736145,1736162,1736209,1736280,1736297,1736299,1736489,1736646,1736703,1736836,1736849,1737104-1737105,1737112,1737117,1737119-1737120,1737155,1737157,1737192,1737280,1737339,1737632,1737664,1737715,1737748,1737785,1737834,1737860,1737903,1737959,1738005,1738007,1738014-1738015,1738018,1738022,1738039,1738043,1738059-1738060,1738147,1738149,1738174-1738175,1738261,1738589,1738623-1738625,1738643,1738816,1738850,1738855,1738946-1738948,1738953-1738954,1738979,1738982,1739079-1739081,1739087,1739113,1739153,1739172,1739176,1739191,1739474,1739492,1739726,1739762,1739775,1739814,1739817-1739818,1739975,1740131,1740324,1740465,1740495,1740508-1740509,1740520,1740535,1740707,1740803,1740810,1740969,1740980,1740991,1740997,1741015,1741033,1741036,1741058,1741060,1741080,1741147,1741159,1741164,1741173,1741181,1741190,1741197,1741202,1741208,1741213,1741221,1741225,1741232,1741409
 
,1741501,1741677,1741892,1741896,1741984,1742023,1742042,1742071,1742090,1742093,1742101,1742105,1742111,1742139,1742146,1742148,1742166,1742181,1742184,1742187,1742246,1742248-1742251,1742263-1742264,1742268,1742276,1742369,1742387,1742448,1742509-1742512,1742917,1742919,1742933,1742975-1742976,1742984,1742986,1743019,1743115,1743117,1743124-1743125,1743134,1743425,1743554,1743679,1743696-1743698,1743700-1743701,1744058,1744064-1744065,1744125,1744149,1744194,1744229,1744270,1744323,1744432,1744684,1744697,1744705,1744713,1744760,1744786,1745083,1745142-1745143,1745145,1745177,1745179-1745180,1745227,1745248,1745254,1745337,1745467,1745473,1745535,1745576,1745735,1745744,1746304,1746306-1746307,1746319,1746327,1746338,1746340-1746341,1746344,1746427,1746441,1746473,1746490,1746492,1746495-1746496,1746499-1746501,1746503-1746507,1746509,1746549,1746551,1746554,1746556,1746558,1746584,1746620,1746649,1746724,1746939,1746989,1747014,1747028,1747035,1747210,1747225,1747234,1747253,1747
 
404,1747506,1747536,1747924,1747980,1747993,1748001,1748253,1748452,1748547,1748629,1748676,1748715,1749287,1749296,1749328,1749373,1749465,1749506,1749508,1749665-1749666,1749763,1749865-1749866,1749898,1749978,1749980,1750011,1750015,1750056,1750480,1750617,1750634,1750692,1750697,1750700,1750703,1750707,1750714,1750718,1750723,1750774,1750899,1750975,1750995,1751061,1751097,1751173,1751438,1751447,1751463,1751702,1752212,1752737,1752745,1753078,1753080,1753358,1753363,1754111,1754140-1754141,1754281,1754310,1754445,1754467,1754494,1754496,1754528,1754532-1754533,1754613,1754714,1754874,1754941,1754944,1754950-1754951,1755005,1755007,1755009,1755132,1755180-1755181,1755185,1755190,1755204-1755206,1755208,1755214,1755224,1755227,1755230,1755629,1755646-1755647,1755650,1755653,1755675,1755680,1755683,1755693,1755717,1755731-1755737,1755812,1755828,1755884,1755890,1755918-1755919,1755942,1755958,1755960,1755970,1755993,1756013,1756019,1756039,1756056,1756083-1756114,1756175,1756288-1
 
756289,1756408-1756410,1756778,1756798,1756878,1756898,1756939,1757123-1757124,1757126,1757128,1757132-1757133,1757136,1757145,1757167-1757168,1757175,1757180,1757182,1757195,1757271,1757278,1757347,1757353-1757354,1757363,1757374,1757399,1757406,1757408,1757485,1757495,1757499,1757527,1757578,1757684,1757722,1757727,1757790,1757799,1757813,1757853,1757883,1757903,1757976,1757997,1758000,1758058,1758072-1758075,1758078-1758079,1758223,1758257,1758261,1758276,1758292,1758369,1758378-1758383,1758421,1758423,1758425-1758427,1758430,1758443,1758448,1758459,1758483,1758486-1758487,1758499,1758525,1758556,1758580,1758582,1758584,1758588,1758842,1759019,1759212,1759224,1759227,1759252,1759274,1759513-1759516,1759611,1759757,1759785-1759790,1760005,1760022,1760109-1760110,1760135,1760200-1760201,1760227,1760300,1760397,1760446,1760454,1760640,1760648,1761057,1761422,1761491,1761498,1761500-1761501,1761550,1761553,1761572,1761574,1761625-1761626,1761628,1761682,1761740,1761752,1762051-176205
 

svn commit: r1838275 - in /tomcat/trunk: java/org/apache/coyote/http2/ test/org/apache/coyote/http2/ webapps/docs/

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 19:16:56 2018
New Revision: 1838275

URL: http://svn.apache.org/viewvc?rev=1838275=rev
Log:
Additional fixes for output corruption of response bodies when writing large 
bodies using asynchronous processing over HTTP/2.

Added:
tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java   (with props)
Modified:
tomcat/trunk/java/org/apache/coyote/http2/Http2AsyncUpgradeHandler.java
tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java
tomcat/trunk/java/org/apache/coyote/http2/Stream.java
tomcat/trunk/test/org/apache/coyote/http2/Http2TestBase.java
tomcat/trunk/webapps/docs/changelog.xml

Modified: 
tomcat/trunk/java/org/apache/coyote/http2/Http2AsyncUpgradeHandler.java
URL: 
http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/coyote/http2/Http2AsyncUpgradeHandler.java?rev=1838275=1838274=1838275=diff
==
--- tomcat/trunk/java/org/apache/coyote/http2/Http2AsyncUpgradeHandler.java 
(original)
+++ tomcat/trunk/java/org/apache/coyote/http2/Http2AsyncUpgradeHandler.java Fri 
Aug 17 19:16:56 2018
@@ -280,7 +280,7 @@ public class Http2AsyncUpgradeHandler ex
 // Reserve as much as possible right away
 int reservation = (sendfile.end - sendfile.pos > 
Integer.MAX_VALUE) ? Integer.MAX_VALUE : (int) (sendfile.end - sendfile.pos);
 sendfile.streamReservation  = 
sendfile.stream.reserveWindowSize(reservation, true);
-sendfile.connectionReservation = 
reserveWindowSize(sendfile.stream, sendfile.streamReservation);
+sendfile.connectionReservation = 
reserveWindowSize(sendfile.stream, sendfile.streamReservation, true);
 } catch (IOException e) {
 return SendfileState.ERROR;
 }
@@ -340,7 +340,7 @@ public class Http2AsyncUpgradeHandler ex
 int reservation = (sendfile.end - sendfile.pos > 
Integer.MAX_VALUE) ? Integer.MAX_VALUE : (int) (sendfile.end - sendfile.pos);
 sendfile.streamReservation = 
sendfile.stream.reserveWindowSize(reservation, true);
 }
-sendfile.connectionReservation = 
reserveWindowSize(sendfile.stream, sendfile.streamReservation);
+sendfile.connectionReservation = 
reserveWindowSize(sendfile.stream, sendfile.streamReservation, true);
 }
 } catch (IOException e) {
 failed (e, sendfile);

Modified: tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java
URL: 
http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java?rev=1838275=1838274=1838275=diff
==
--- tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java 
(original)
+++ tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java Fri Aug 
17 19:16:56 2018
@@ -723,7 +723,7 @@ class Http2UpgradeHandler extends Abstra
 }
 
 
-int reserveWindowSize(Stream stream, int reservation) throws IOException {
+int reserveWindowSize(Stream stream, int reservation, boolean block) 
throws IOException {
 // Need to be holding the stream lock so releaseBacklog() can't notify
 // this thread until after this thread enters wait()
 int allocation = 0;
@@ -775,12 +775,16 @@ class Http2UpgradeHandler extends Abstra
 }
 }
 if (allocation == 0) {
-try {
-stream.wait();
-} catch (InterruptedException e) {
-throw new IOException(sm.getString(
-
"upgradeHandler.windowSizeReservationInterrupted", connectionId,
-stream.getIdentifier(), 
Integer.toString(reservation)), e);
+if (block) {
+try {
+stream.wait();
+} catch (InterruptedException e) {
+throw new IOException(sm.getString(
+
"upgradeHandler.windowSizeReservationInterrupted", connectionId,
+stream.getIdentifier(), 
Integer.toString(reservation)), e);
+}
+} else {
+return 0;
 }
 }
 } while (allocation == 0);

Modified: tomcat/trunk/java/org/apache/coyote/http2/Stream.java
URL: 
http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/coyote/http2/Stream.java?rev=1838275=1838274=1838275=diff
==
--- tomcat/trunk/java/org/apache/coyote/http2/Stream.java (original)
+++ tomcat/trunk/java/org/apache/coyote/http2/Stream.java Fri Aug 17 19:16:56 
2018
@@ -718,6 +718,7 

svn commit: r1838274 - in /tomcat/tc8.5.x/trunk: ./ java/org/apache/coyote/ java/org/apache/coyote/http2/

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 19:16:36 2018
New Revision: 1838274

URL: http://svn.apache.org/viewvc?rev=1838274=rev
Log:
Additional debug logging

Modified:
tomcat/tc8.5.x/trunk/   (props changed)
tomcat/tc8.5.x/trunk/java/org/apache/coyote/AbstractProcessorLight.java
tomcat/tc8.5.x/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java
tomcat/tc8.5.x/trunk/java/org/apache/coyote/http2/LocalStrings.properties
tomcat/tc8.5.x/trunk/java/org/apache/coyote/http2/StreamProcessor.java

Propchange: tomcat/tc8.5.x/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Fri Aug 17 19:16:36 2018
@@ -1,2 +1,2 @@
 /tomcat/tc8.0.x/trunk:1809644
-/tomcat/trunk:1734785,1734799,1734845,1734928,1735041,1735044,1735480,1735577,1735597,1735599-1735600,1735615,1736145,1736162,1736209,1736280,1736297,1736299,1736489,1736646,1736703,1736836,1736849,1737104-1737105,1737112,1737117,1737119-1737120,1737155,1737157,1737192,1737280,1737339,1737632,1737664,1737715,1737748,1737785,1737834,1737860,1737903,1737959,1738005,1738007,1738014-1738015,1738018,1738022,1738039,1738043,1738059-1738060,1738147,1738149,1738174-1738175,1738261,1738589,1738623-1738625,1738643,1738816,1738850,1738855,1738946-1738948,1738953-1738954,1738979,1738982,1739079-1739081,1739087,1739113,1739153,1739172,1739176,1739191,1739474,1739492,1739726,1739762,1739775,1739814,1739817-1739818,1739975,1740131,1740324,1740465,1740495,1740508-1740509,1740520,1740535,1740707,1740803,1740810,1740969,1740980,1740991,1740997,1741015,1741033,1741036,1741058,1741060,1741080,1741147,1741159,1741164,1741173,1741181,1741190,1741197,1741202,1741208,1741213,1741221,1741225,1741232,1741409
 
,1741501,1741677,1741892,1741896,1741984,1742023,1742042,1742071,1742090,1742093,1742101,1742105,1742111,1742139,1742146,1742148,1742166,1742181,1742184,1742187,1742246,1742248-1742251,1742263-1742264,1742268,1742276,1742369,1742387,1742448,1742509-1742512,1742917,1742919,1742933,1742975-1742976,1742984,1742986,1743019,1743115,1743117,1743124-1743125,1743134,1743425,1743554,1743679,1743696-1743698,1743700-1743701,1744058,1744064-1744065,1744125,1744149,1744194,1744229,1744270,1744323,1744432,1744684,1744697,1744705,1744713,1744760,1744786,1745083,1745142-1745143,1745145,1745177,1745179-1745180,1745227,1745248,1745254,1745337,1745467,1745473,1745535,1745576,1745735,1745744,1746304,1746306-1746307,1746319,1746327,1746338,1746340-1746341,1746344,1746427,1746441,1746473,1746490,1746492,1746495-1746496,1746499-1746501,1746503-1746507,1746509,1746549,1746551,1746554,1746556,1746558,1746584,1746620,1746649,1746724,1746939,1746989,1747014,1747028,1747035,1747210,1747225,1747234,1747253,1747
 
404,1747506,1747536,1747924,1747980,1747993,1748001,1748253,1748452,1748547,1748629,1748676,1748715,1749287,1749296,1749328,1749373,1749465,1749506,1749508,1749665-1749666,1749763,1749865-1749866,1749898,1749978,1749980,1750011,1750015,1750056,1750480,1750617,1750634,1750692,1750697,1750700,1750703,1750707,1750714,1750718,1750723,1750774,1750899,1750975,1750995,1751061,1751097,1751173,1751438,1751447,1751463,1751702,1752212,1752737,1752745,1753078,1753080,1753358,1753363,1754111,1754140-1754141,1754281,1754310,1754445,1754467,1754494,1754496,1754528,1754532-1754533,1754613,1754714,1754874,1754941,1754944,1754950-1754951,1755005,1755007,1755009,1755132,1755180-1755181,1755185,1755190,1755204-1755206,1755208,1755214,1755224,1755227,1755230,1755629,1755646-1755647,1755650,1755653,1755675,1755680,1755683,1755693,1755717,1755731-1755737,1755812,1755828,1755884,1755890,1755918-1755919,1755942,1755958,1755960,1755970,1755993,1756013,1756019,1756039,1756056,1756083-1756114,1756175,1756288-1
 
756289,1756408-1756410,1756778,1756798,1756878,1756898,1756939,1757123-1757124,1757126,1757128,1757132-1757133,1757136,1757145,1757167-1757168,1757175,1757180,1757182,1757195,1757271,1757278,1757347,1757353-1757354,1757363,1757374,1757399,1757406,1757408,1757485,1757495,1757499,1757527,1757578,1757684,1757722,1757727,1757790,1757799,1757813,1757853,1757883,1757903,1757976,1757997,1758000,1758058,1758072-1758075,1758078-1758079,1758223,1758257,1758261,1758276,1758292,1758369,1758378-1758383,1758421,1758423,1758425-1758427,1758430,1758443,1758448,1758459,1758483,1758486-1758487,1758499,1758525,1758556,1758580,1758582,1758584,1758588,1758842,1759019,1759212,1759224,1759227,1759252,1759274,1759513-1759516,1759611,1759757,1759785-1759790,1760005,1760022,1760109-1760110,1760135,1760200-1760201,1760227,1760300,1760397,1760446,1760454,1760640,1760648,1761057,1761422,1761491,1761498,1761500-1761501,1761550,1761553,1761572,1761574,1761625-1761626,1761628,1761682,1761740,1761752,1762051-176205
 

[Bug 62582] Please add OWASP Dependency Check to the build for all Tomcat componets

2018-08-17 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=62582

--- Comment #8 from Christopher Schultz  ---
(In reply to mgrigorov from comment #4)
> As Mark explained there is not much to check anyway because Tomcat doesn't
> have dependencies.

Well... that's not entirely true. Tomcat does rely on:

1. commons-dbcp[2]
2. commons-fileupload
3. ecj
4. commons-daemon
5. tcnative (technically)

But my guess is that 1, 2 above wouldn't be detected because they have been
modified from their original and re-packaged with "Tomcat" as the product name
(instead of e.g. commons-dbcp).

OWASP isn't very good at detecting binary dependencies, so it wouldn't detect
5.

I'm not saying this is a great idea, but I'm not saying it's totally worthless,
either.

I think having an ant target that is runnable by anyone who chooses to run it
would be a good thing.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 62626] Tomcat 9.0.10 APR/Native crashes

2018-08-17 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626

--- Comment #12 from Christopher Schultz  ---
(In reply to jan.pfeifer from comment #11)
> (In reply to Christopher Schultz from comment #10)
> > Have a look at this page:
> > https://wiki.apache.org/tomcat/FAQ/KnownIssues#ImageIOIssues
> 
> If I undestand correctly there is only problem when using ImageIO.write with
> response output stream. I do not use it that way. I write with ImageIO to
> filesystem. Then, when image is ready, I read it again and write it to
> response stream via Buffered IN/Out stream.

Hmm, yes, this was in cases where ImageIO was being used to write to the
servlet output stream. So it looks like that isn't the problem.

> > Try going back up to Java 10 and wrapping your streaming-ImageIO uses in the
> > class shown in that wiki page and re-running. If the crashes stop, we can go
> > ahead and blame them on ImageIO, but really tcnative should not crash in
> > this scenario... I'd expect that you'd just get lots of IOExceptions
> > instead. So tcnative isn't doing its job under this circumstance.
> 
> I left if for weekend with Java 8 to see what happens. Yes there are many
> IOExceptions already. No crash so far.

Just to confirm: this is Java 8 with APR+OpenSSL, not NIO+OpenSSL, correct?

If that's the case, there might be some kind of incompatibility between
tcnative and Java 9. That would be very surprising to me, but that seems to be
where the data are leading us.

> > Unfortunately, I don't have  win32 debugging environment available to me, so
> > I'm not able to turn the 0xe6fc9 offset into a meaningful line of code to
> > investigate. It's obviously somewhere in the tcnative implementation of
> > SSLSocket.handshake (or something it calls). That function has ~100 lines of
> > code in it, but I'm guessing it's crashing towards the beginning, probably
> > on a read.
> > 
> > Can you use a DEBUG build of tcnative to get a better crash report? I'm used
> > to seeing more of a stack trace in native crash dumps.
> 
> I will try to get DEBUG build on monday. But there is a lot of other debug
> info already. As I posted earlier i can attach it. Tomcat generated very
> long bug report from which i posted the top part and sysinfo part, and
> several gigs big mdmp file. Do you want to see them or we stick to DEBUG
> build of tcnative plan?

The Java stack traces are less important than the native stack trace. There was
only a single item in the native stack trace you already posted. Can you post
the part of the native report that is labelled "-- T H R E A D ---"?

That should include a lot more relevant information.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 62626] Tomcat 9.0.10 APR/Native crashes

2018-08-17 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626

--- Comment #11 from jan.pfei...@centrum.cz ---
(In reply to Christopher Schultz from comment #10)
> Have a look at this page:
> https://wiki.apache.org/tomcat/FAQ/KnownIssues#ImageIOIssues

If I undestand correctly there is only problem when using ImageIO.write with
response output stream. I do not use it that way. I write with ImageIO to
filesystem. Then, when image is ready, I read it again and write it to response
stream via Buffered IN/Out stream.

> Try going back up to Java 10 and wrapping your streaming-ImageIO uses in the
> class shown in that wiki page and re-running. If the crashes stop, we can go
> ahead and blame them on ImageIO, but really tcnative should not crash in
> this scenario... I'd expect that you'd just get lots of IOExceptions
> instead. So tcnative isn't doing its job under this circumstance.

I left if for weekend with Java 8 to see what happens. Yes there are many
IOExceptions already. No crash so far.

> Unfortunately, I don't have  win32 debugging environment available to me, so
> I'm not able to turn the 0xe6fc9 offset into a meaningful line of code to
> investigate. It's obviously somewhere in the tcnative implementation of
> SSLSocket.handshake (or something it calls). That function has ~100 lines of
> code in it, but I'm guessing it's crashing towards the beginning, probably
> on a read.
> 
> Can you use a DEBUG build of tcnative to get a better crash report? I'm used
> to seeing more of a stack trace in native crash dumps.

I will try to get DEBUG build on monday. But there is a lot of other debug info
already. As I posted earlier i can attach it. Tomcat generated very long bug
report from which i posted the top part and sysinfo part, and several gigs big
mdmp file. Do you want to see them or we stick to DEBUG build of tcnative plan?

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Securing temporary file-uploads

2018-08-17 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

All,

A presentation at DEFCON[1] last week showed how many Android
applications were improperly using shared external storage on Android
devices and could be vulnerable due to improper handling of temporary
files.

This was your basic:

1. Download a file from a trusted source (e.g. properly-authenticated,
encrypted web connection)
2. Buffer the file in temporary storage
3. Read the file back from temporary storage

type setup.

The problem is that step 2b. might be "an attacker modifies the file".

If step 4 is "use the file to update your own application", and the
storage system is untrustworthy (which is the case on Android, since
it's shared among all apps running on the device), then there is an
opportunity to exploit that application.

This got me to thinking about how Tomcat handles file uploads. Since
Servlet 3.0, containers must provide the facility and Tomcat uses
commons-fileupload to handle most of the heavy-lifting. One feature of
that facility is the ability to buffer the file on a local disk if it
exceeds some defined size -- to avoid creating large in-memory buffers
for file uploads.

So Tomcat does some form of the steps 1-3 above, except that the file
isn't being downloaded... it's being uploaded. Now, it's really up to
the application developer to be sure that the user should be allowed
to upload a file, but assuming that user has been authenticated, then
the application should be able to trust that Tomcat hasn't been
subverted during steps 2-3.

If the filesystem is hostile (yes, I know you have lots of problems if
that's the case), then Tomcat might end up not delivering the same
bytes to the application that were originally uploaded by the user.

I'm curious whether anyone cares to look at this scenario in order to
mitigate it. I can think of a few ways to mitigate such a potential
vulnerability:

1. Tomcat computes a signature of the file as it's being written-out,
computes the signature of the file as it's being read back-in for the
application, and throws an exception if the file appears to be
corrupted. Problems with this solution include not being able to
detect the problem until most of the bytes have already been sent to
the application.

2. Tomcat encrypts the file as its being written to the disk with a
temporary symmetric key. Problems with this solution are additional
resources (CPU) required for encryption.

The LOE for either of the above doesn't seem very high.

Is there any appetite for hardening of this type in Tomcat? Situations
where the filesystem (or other programs running on the server) is
untrustworthy are probably few and far-between, but there are indeed
environments where applications are running under a security manager
because those applications are untrusted.

I didn't check to see whether Tomcat segregates temporary files for
uploads between different applications, but I can see a scenario or
two where Tomcat might want to protect an application from having its
uploads tampered with.

Thanks,
- -chris

[1]
https://blog.checkpoint.com/2018/08/12/man-in-the-disk-a-new-attack-surf
ace-for-android-apps/
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlt23PwACgkQHPApP6U8
pFhyWQ/9E19l0tN1T37T/GPo3HsLZ+81MlJ/XKY+LrCQsKLG7Q+q1RnIOIjf3PnL
uCy12JdH9eIZK96w35RXp6CP8H4t3FwaDEAoajUr1WYiqxi2L44cLxXd84gw0KGE
IwF+yaY91YDIqMLnY7vf8Qq4rFygjML4o9Lu2PqffIdzgYbarWIYyhlx9yKX5FMe
1w+QD5qLvjVRWYig3WN5e+np0bDFZH6YAqQumQWDTLIFZrPq7BE6AlXX8cSXOhN8
LLvbULsusJYrDxiS2yximiyDema69vUfKutFKlRCgW0F+feyRSqy5RjjlUgY4KJ6
0gdwXKaHC4L7SaifKwjABQgIt7SIZPOTIaVIDEf+A2Ge+TcH3jiHJ/djcrY+MvoO
73ZOVr4sf0H3hKjDzDB3METpomP40CdjMhhgE/8WinORl/ZBJ6gPcP+wwsgEMo+I
wbRnF/mraBaXHNuBZtx/gFciuUwUmyNBw7s2glYuDpC4cPa5r2naKdDz0an82ESu
9g1j2HYOW7htCCEnNYRTdeQPso3i5hkdr7rIW1pyRWspeoI5I5HxjQeIIrEYFRYa
CZ1zmJXqezITvqS/yxPmkijnmT45kvtiVld4RROFwxNEmXghY+iDZ2pxvnZozhoI
y2Uzu//8NFEeFNCyIV3iP9b3io/uX81iF51K5MqW/wwPyLlYLAg=
=U/5d
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 62626] Tomcat 9.0.10 APR/Native crashes

2018-08-17 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626

--- Comment #10 from Christopher Schultz  ---
Have a look at this page:
https://wiki.apache.org/tomcat/FAQ/KnownIssues#ImageIOIssues

Try going back up to Java 10 and wrapping your streaming-ImageIO uses in the
class shown in that wiki page and re-running. If the crashes stop, we can go
ahead and blame them on ImageIO, but really tcnative should not crash in this
scenario... I'd expect that you'd just get lots of IOExceptions instead. So
tcnative isn't doing its job under this circumstance.

(n.b.: http/2 does not require Java 9.)

Unfortunately, I don't have  win32 debugging environment available to me, so
I'm not able to turn the 0xe6fc9 offset into a meaningful line of code to
investigate. It's obviously somewhere in the tcnative implementation of
SSLSocket.handshake (or something it calls). That function has ~100 lines of
code in it, but I'm guessing it's crashing towards the beginning, probably on a
read.

Can you use a DEBUG build of tcnative to get a better crash report? I'm used to
seeing more of a stack trace in native crash dumps.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: [VOTE] Release Apache Tomcat 8.5.33

2018-08-17 Thread Violeta Georgieva
Hi,

На нд, 12.08.2018 г. в 11:43 ч. Mark Thomas  написа:
>
> The proposed Apache Tomcat 8.5.33 release is now available for voting.
>
> The major changes compared to the 8.5.32 release are:
>
> - Fix issues with Servlet asynchronous listeners when using the
>   asynchronous Servlet API in conjunction with HTTP/2.
>
> - When pre-compiling with JspC, report all compilation errors rather
>   than stopping after the first error. A new option -failFast can be
>   used to restore the previous behaviour of stopping after the first
>   error.
>   Based on a patch provided by Marc Pompl.
>
> - Make the Jasper (JSP Engine) Java file generation process
>   multi-threaded. By default, one thread will be used per core.
>   Based on a patch by Dan Fabulich.
>
> Along with lots of other bug fixes and improvements.
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.33/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1192/
> The svn tag is:
> http://svn.apache.org/repos/asf/tomcat/tc8.5.x/tags/TOMCAT_8_5_33/
>
> The proposed 8.5.33 release is:
> [ ] Broken - do not release
> [X] Stable - go ahead and release as 8.5.33

+1

Regards,
Violeta


buildbot success in on tomcat-trunk

2018-08-17 Thread buildbot
The Buildbot has detected a restored build on builder tomcat-trunk while 
building . Full details are available at:
https://ci.apache.org/builders/tomcat-trunk/builds/3526

Buildbot URL: https://ci.apache.org/

Buildslave for this Build: silvanus_ubuntu

Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-commit' 
triggered this build
Build Source Stamp: [branch tomcat/trunk] 1838258
Blamelist: markt

Build succeeded!

Sincerely,
 -The Buildbot




-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



svn commit: r28803 - /dev/tomcat/tomcat-9/v9.0.11/ /release/tomcat/tomcat-9/v9.0.11/

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 13:13:35 2018
New Revision: 28803

Log:
Release Apache Tomcat 9.0.11

Added:
release/tomcat/tomcat-9/v9.0.11/
  - copied from r28802, dev/tomcat/tomcat-9/v9.0.11/
Removed:
dev/tomcat/tomcat-9/v9.0.11/


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



svn commit: r1838258 - /tomcat/trunk/webapps/docs/changelog.xml

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 13:00:29 2018
New Revision: 1838258

URL: http://svn.apache.org/viewvc?rev=1838258=rev
Log:
Add the release date for 9.0.11

Modified:
tomcat/trunk/webapps/docs/changelog.xml

Modified: tomcat/trunk/webapps/docs/changelog.xml
URL: 
http://svn.apache.org/viewvc/tomcat/trunk/webapps/docs/changelog.xml?rev=1838258=1838257=1838258=diff
==
--- tomcat/trunk/webapps/docs/changelog.xml (original)
+++ tomcat/trunk/webapps/docs/changelog.xml Fri Aug 17 13:00:29 2018
@@ -67,7 +67,7 @@
 
   
 
-
+
   
 
   



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[VOTE][RESULT] Release Apache Tomcat 9.0.11

2018-08-17 Thread Mark Thomas
The following votes were cast:

Binding:
+1: markt, fschumacher, violetagg

No other votes were cast.

Thanks to everyone who contributed to this release.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 62626] Tomcat 9.0.10 APR/Native crashes

2018-08-17 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626

--- Comment #9 from jan.pfei...@centrum.cz ---
(In reply to Mark Thomas from comment #8)
> You mentioned imageio. I'm not sure at what point you stooped using it but
> there have been consistent reports over the years of imageio holding on to
> references to I/O streams and using them long after they should have been
> finished with. This has been the cause of APR crashes in the past. e.g.:
> https://markmail.org/message/xr3c5dixxzetpbez

Yes, you are right, that was the reason. But there were several chrashes after
the change. First was from https://bitbucket.org/luciad/webp-imageio, the
second is from https://developers.google.com/speed/webp/download
(img2webp.exe).

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



buildbot failure in on tomcat-trunk

2018-08-17 Thread buildbot
The Buildbot has detected a new failure on builder tomcat-trunk while building 
. Full details are available at:
https://ci.apache.org/builders/tomcat-trunk/builds/3525

Buildbot URL: https://ci.apache.org/

Buildslave for this Build: silvanus_ubuntu

Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-commit' 
triggered this build
Build Source Stamp: [branch tomcat/trunk] 1838243
Blamelist: markt

BUILD FAILED: failed compile_1

Sincerely,
 -The Buildbot




-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 62633] New: WsSession leaks if cache Session are used for group sending

2018-08-17 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=62633

Bug ID: 62633
   Summary: WsSession leaks if cache Session are used for group
sending
   Product: Tomcat 8
   Version: 8.5.31
  Hardware: PC
Status: NEW
  Severity: normal
  Priority: P2
 Component: WebSocket
  Assignee: dev@tomcat.apache.org
  Reporter: 1005136...@qq.com
  Target Milestone: 

Created attachment 36098
  --> https://bz.apache.org/bugzilla/attachment.cgi?id=36098=edit
code and war

public class MySessionContainer {

private final static Map WS_SESSION_USER_MAP = new
ConcurrentHashMap<>();

public static void addClient(String id, WsSessionUser user) {
WS_SESSION_USER_MAP.put(id, user);
}

public static WsSessionUser getClient(String id) {
return WS_SESSION_USER_MAP.get(id);
}

public static WsSessionUser removeClient(String id) {
return WS_SESSION_USER_MAP.remove(id);
}
}


public class WsSessionUser {

private Session session;

private ThreadPoolExecutor threadPoolExecutor = new
ThreadPoolExecutor(1, 1, 1, TimeUnit.MINUTES,
new LinkedBlockingQueue<>(2));

public WsSessionUser(Session session) {
this.session = session;
}

public void sendMessageByPool(String message) {
try {
if (!threadPoolExecutor.isShutdown()) {
threadPoolExecutor.execute(() ->
sendMessage(message));
}
} catch (RejectedExecutionException e) {
threadPoolExecutor.shutdownNow();
try {
if (session.isOpen()) {
session.close();
}
} catch (IOException ignored) {
}
}
}

public void sendMessage(String message) {
try {
if (session.isOpen()) {
session.getBasicRemote().sendText(message);
}
} catch (IOException e) {
e.printStackTrace();
}
}

public ThreadPoolExecutor getThreadPoolExecutor() {
return threadPoolExecutor;
}

public Session getSession() {
return session;
}


}


@ServerEndpoint("/ws/server")
public class MyWebSocketServer {

@OnOpen
public void onOpen(Session session) {
System.out.println("server OnOpen");
MySessionContainer.addClient(session.getId(), new
WsSessionUser(session));
}

@OnClose
public void onClose(Session session) {
System.out.println("server OnClose");
WsSessionUser wsSessionUser =
MySessionContainer.removeClient(session.getId());
if (wsSessionUser != null) {
ThreadPoolExecutor threadPoolExecutor =
wsSessionUser.getThreadPoolExecutor();
if (!threadPoolExecutor.isShutdown()) {
threadPoolExecutor.shutdownNow();
}
try {
wsSessionUser.getSession().close();
} catch (IOException e) {
e.printStackTrace();
}
}
}

@OnMessage
public void onMessage(String message, Session session) {
System.out.println("server OnMessage");
}

@OnError
public void onError(Session session, Throwable error) {
System.out.println("server OnError");
}

}

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



svn commit: r1838243 - in /tomcat/trunk/java/org/apache/coyote: AbstractProcessorLight.java http2/Http2UpgradeHandler.java http2/LocalStrings.properties http2/StreamProcessor.java

2018-08-17 Thread markt
Author: markt
Date: Fri Aug 17 08:56:50 2018
New Revision: 1838243

URL: http://svn.apache.org/viewvc?rev=1838243=rev
Log:
Additional debug logging

Modified:
tomcat/trunk/java/org/apache/coyote/AbstractProcessorLight.java
tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java
tomcat/trunk/java/org/apache/coyote/http2/LocalStrings.properties
tomcat/trunk/java/org/apache/coyote/http2/StreamProcessor.java

Modified: tomcat/trunk/java/org/apache/coyote/AbstractProcessorLight.java
URL: 
http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/coyote/AbstractProcessorLight.java?rev=1838243=1838242=1838243=diff
==
--- tomcat/trunk/java/org/apache/coyote/AbstractProcessorLight.java (original)
+++ tomcat/trunk/java/org/apache/coyote/AbstractProcessorLight.java Fri Aug 17 
08:56:50 2018
@@ -70,16 +70,20 @@ public abstract class AbstractProcessorL
 state = SocketState.CLOSED;
 }
 
-if (state != SocketState.CLOSED && isAsync()) {
-state = asyncPostProcess();
-}
-
 if (getLog().isDebugEnabled()) {
 getLog().debug("Socket: [" + socketWrapper +
 "], Status in: [" + status +
 "], State out: [" + state + "]");
 }
 
+if (state != SocketState.CLOSED && isAsync()) {
+state = asyncPostProcess();
+if (getLog().isDebugEnabled()) {
+getLog().debug("Socket: [" + socketWrapper +
+"], State after async post processing: [" + state 
+ "]");
+}
+}
+
 if (dispatches == null || !dispatches.hasNext()) {
 // Only returns non-null iterator if there are
 // dispatches to process.

Modified: tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java
URL: 
http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java?rev=1838243=1838242=1838243=diff
==
--- tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java 
(original)
+++ tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java Fri Aug 
17 08:56:50 2018
@@ -818,11 +818,19 @@ class Http2UpgradeHandler extends Abstra
 }
 Response coyoteResponse = ((Stream) 
stream).getCoyoteResponse();
 if (coyoteResponse.getWriteListener() == null) {
+if (log.isDebugEnabled()) {
+log.debug(sm.getString("upgradeHandler.notifyAll",
+connectionId, stream.getIdentifier()));
+}
 // Blocking, so use notify to release StreamOutputBuffer
 synchronized (stream) {
 stream.notifyAll();
 }
 } else {
+if (log.isDebugEnabled()) {
+log.debug(sm.getString("upgradeHandler.dispatchWrite",
+connectionId, stream.getIdentifier()));
+}
 // Non-blocking so dispatch
 coyoteResponse.action(ActionCode.DISPATCH_WRITE, null);
 // Need to explicitly execute dispatches on the

Modified: tomcat/trunk/java/org/apache/coyote/http2/LocalStrings.properties
URL: 
http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/coyote/http2/LocalStrings.properties?rev=1838243=1838242=1838243=diff
==
--- tomcat/trunk/java/org/apache/coyote/http2/LocalStrings.properties [UTF-8] 
(original)
+++ tomcat/trunk/java/org/apache/coyote/http2/LocalStrings.properties [UTF-8] 
Fri Aug 17 08:56:50 2018
@@ -104,6 +104,7 @@ stream.outputBuffer.flush.debug=Connecti
 
 streamProcessor.error.connection=Connection [{0}], Stream [{1}], An error 
occurred during processing that was fatal to the connection
 streamProcessor.error.stream=Connection [{0}], Stream [{1}], An error occurred 
during processing that was fatal to the stream
+streamProcessor.flushBufferedWrite.entry=Connection [{0}], Stream [{1}], 
Flushing buffered writes
 streamProcessor.service.error=Error during request processing
 
 streamStateMachine.debug.change=Connection [{0}], Stream [{1}], State changed 
from [{2}] to [{3}]
@@ -115,12 +116,14 @@ upgradeHandler.allocate.left=Connection
 upgradeHandler.allocate.recipient=Connection [{0}], Stream [{1}], potential 
recipient [{2}] with weight [{3}]
 upgradeHandler.connectionError=Connection error
 upgradeHandler.dependency.invalid=Connection [{0}], Stream [{1}], Streams may 
not depend on themselves
+upgradeHandler.dispatchWrite=Connection [{0}], Stream [{1}], Dispatching to 
container thread for async write
 

[Bug 62626] Tomcat 9.0.10 APR/Native crashes

2018-08-17 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626

--- Comment #8 from Mark Thomas  ---
You mentioned imageio. I'm not sure at what point you stooped using it but
there have been consistent reports over the years of imageio holding on to
references to I/O streams and using them long after they should have been
finished with. This has been the cause of APR crashes in the past. e.g.:
https://markmail.org/message/xr3c5dixxzetpbez

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 62626] Tomcat 9.0.10 APR/Native crashes

2018-08-17 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626

--- Comment #7 from jan.pfei...@centrum.cz ---
(In reply to Christopher Schultz from comment #6)

> That's kind of par for the course, isn't it? Any IO operation can fail for
> any reason. If you don't catch it and handle it, Tomcat will (eventually)
> log it. What else did you expect to happen?

I was only surprised by number of different IO exceptions. For some (Stream is
not writable) it is not very clear what do they mean.

> A "client abort" exception happens when the client makes a request and then
> hangs-up the phone before you complete the response. It's fairly common and
> I wouldn't really expect tcnative to crash under that circumstance, but it's
> certainly possible.
> 
> In general, ClientAbortExceptions can safely be completely ignored, so don't
> worry too much about the fact that they are happening. On a busy site with
> large responses (e.g. images) I'd expect lots of them.

I undestand that. As I stated above it was only mix of different IOException
types 
and quantity what confused me.

> Can you try one more thing for me? Downgrade to Java 1.8.0_whatever and
> revert your configuration to use APR+tcnative again and let us know if the
> crashes (actual segfaults that kill the JVM) continue.

I managed the downgrade. Alas lots of traffic ceased at the moment and server
seems stable. I removed HTTP/2 upgrade as I understand it needs Java 9. Or am I
wrong?


> But let's take your ClientAbortException stuff onto the user list to see if
> we can't find maybe some more efficient ways to get your images generated.
> That synchronized block looks suspicious to me, and creating new threads all
> the time is likely to lead to instability.

In first version i wasnt using threads at all. It seemed unlikely to get
several concurrent request for one image. If it would happened the later one
could simply overwrite image or respond with error. But since the crashes
started i suspected thread concurency to be a problem. 

New threads are only spawned when images is requested for the first time.
Images is downloaded from different machine, resized and webp version is
created. WebP is now created with an external program. I was using native lib
and imageio before.

I use threads and ConcurentHashMap only to ensure that incoming
requests will wait until image processing is finished. This only applies for
requests for the same image. Others are free to get cached image or to start
new resizing thread. Other option would be to allow only one image process at
time, but it would have negative performance impact i guess. 

Another strange thing i got into was out of memory error (4GB) yesterday
evening. No obvious reason of the problem. I have deleted image cache, so they
are recreated now. Everything runs well.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org