Addition to the previous comment : we want exactly the behaviour that is 
described in  this answer
https://stackoverflow.com/a/46142976/547065 , under “Level-triggered 
EPOLLONESHOT”
thus we use the EPOLL flags that we use.

As original author remarks, on Linux, the socket is not physically removed from 
set after one-shot.  Just event notification is disabled, until reenabled 
again,  with epoll_ctl / EPOLL_CTL_MOD . This should be a really fast syscall, 
because it does almost nothing.


From: Vladislav Vaintroub<mailto:vvaintr...@gmail.com>
Sent: Thursday, 18 April 2019 19:19
To: REIX, Tony<mailto:tony.r...@atos.net>; 
maria-developers@lists.launchpad.net<mailto:maria-developers@lists.launchpad.net>
Cc: GUESNET, ETIENNE (ext)<mailto:etienne.guesnet.exter...@atos.net>
Subject: RE: [Maria-developers] MariaDB 10.4 on AIX. Issue with: 
HAVE_POOL_OF_THREADS

Hi Tony,

MariaDB uses EPOLLETT| EPOLLONESHOT on Linux.

By now,  I forgot which EPOLLXXX flag does what exactly.

The intention  is to remove the socket from the “pollset” once we determine 
there is some data to read. Then server executes query, and after that 
threadpool puts socket  back to pollset. The reason to remove socket from 
pollset is that we want to prevent multiple worker threads handling the same 
connection at the same time.

Also, when query is finished, and we add the socket back to the set, and there 
is already some new data in the socket (e.g new query),  threadpool wants to be 
informed about this fact.

This way it works on all platforms in sql/threadpool_generic.cc, and the flags 
on Linux were chosen to satisfy the above conditions.



From: REIX, Tony <tony.r...@atos.net>
Sent: Thursday, April 18, 2019 6:03:06 PM
To: Vladislav Vaintroub; maria-developers@lists.launchpad.net
Cc: GUESNET, ETIENNE (ext)
Subject: RE: [Maria-developers] MariaDB 10.4 on AIX. Issue with: 
HAVE_POOL_OF_THREADS


Hi Wlad,



Thanks for fixing this issue. :-)



About thread-pool, the implementation of IOCP on AIX has limitations. That 
works ONLY with Asynchronous I/O (AIO).



On AIX, we have the pollset API. Within our port of Go on AIX, we did not use 
pollset since Go wants netpoll to be edge-triggered (epoll with EPOLLET flag) 
and not level-triggered. However, this is very unefficient to implement 
edge-triggerered behavior with pollset (one has to continuously modify the 
pollset, destroying the performance).



So, we have to understand if MariaDB requires edge-triggered or 
level-triggered. Do you know?



Thanks/Regards,


Cordialement,

Tony Reix

tony.r...@atos.net

ATOS / Bull SAS
ATOS Expert
IBM Coop Architect & Technical Leader
Office : +33 (0) 4 76 29 72 67
1 rue de Provence - 38432 Échirolles - France
www.atos.net<https://mail.ad.bull.net/owa/redir.aspx?C=PvphmPvCZkGrAgHVnWGsdMcDKgzl_dEIsM6rX0g4u4v8V81YffzBGkWrtQeAXNovd3ttkJL8JIc.&URL=http%3a%2f%2fwww.atos.net%2f>


De : Vladislav Vaintroub <vvaintr...@gmail.com>
Envoyé : jeudi 18 avril 2019 11:54
À : REIX, Tony; maria-developers@lists.launchpad.net
Objet : RE: [Maria-developers] MariaDB 10.4 on AIX. Issue with: 
HAVE_POOL_OF_THREADS


Hi Tony,



I fixed this in 
https://jira.mariadb.org/browse/MDEV-19274<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjira.mariadb.org%2Fbrowse%2FMDEV-19274&data=02%7C01%7Ctony.reix%40atos.net%7Cb5e9b806e428485b785c08d6c3e3e1b8%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C636911780844412005&sdata=XWFIvR5T9zP4eR%2F5Jqwb1%2FoFvr809azhFplEtSGy4R4%3D&reserved=0>
 .



I do not think it makes a lot of sense to implement generic, poll()-based 
threadpool . epoll-like APIs exists almost everywhere, except  very exotic (for 
MariaDB) platforms we do not build on.



I googled, and I found that AIX has  Windows-like IOCP with 
GetQueuedCompletionStatus/CreateIoCompletionPort etc. Maybe, it does not behave 
exactly like on Windows, but perhaps you can figure out what’s the difference 
and  what is required to port that code.



Regards,

Wlad



From: REIX, Tony<mailto:tony.r...@atos.net>
Sent: Tuesday, 16 April 2019 11:26
To: 
maria-developers@lists.launchpad.net<mailto:maria-developers@lists.launchpad.net>
Subject: [Maria-developers] MariaDB 10.4 on AIX. Issue with: 
HAVE_POOL_OF_THREADS



Hi,

In my environment (AIX & MariaDB v10.4.3 or 10.4.4), I have an issue with 
HAVE_POOL_OF_THREADS.

When building, I have the following trace and error (full line of g++ 
compilation at end of email) :

[ 44%] Linking CXX executable explain_filename-t
/opt/freeware/bin/cmake -E cmake_link_script 
CMakeFiles/explain_filename-t.dir/link.txt --verbose=1
/opt/freeware/bin/g++ ...........  -o explain_filename-t ....
ld: 0711-317 ERROR: Undefined symbol: .TP_pool_generic::TP_pool_generic()
ld: 0711-317 ERROR: Undefined symbol: .TP_pool_generic::init()

This seems to be due to the following lines of the file:
sql/CMakeLists.txt :

  IF (CMAKE_SYSTEM_NAME MATCHES "Linux" OR
      CMAKE_SYSTEM_NAME MATCHES "Windows" OR
      CMAKE_SYSTEM_NAME MATCHES "SunOS" OR
      HAVE_KQUEUE)
   ADD_DEFINITIONS(-DHAVE_POOL_OF_THREADS)
   IF(WIN32)
     SET(SQL_SOURCE ${SQL_SOURCE} threadpool_win.cc)
   ENDIF()
   SET(SQL_SOURCE ${SQL_SOURCE} threadpool_generic.cc)
  ENDIF()

In case the Operating System is not Linux, Windows, or SunOS, then there is NO  
-DHAVE_POOL_OF_THREADS  and the file: threadpool_generic.cc is NOT compiled.

However, the C++ code of MariaDB still looks at least for 2 routines of 
threadpool_generic.cc  :
  TP_pool_generic::TP_pool_generic()
  TP_pool_generic::init()
but these routines are not available since threadpool_generic.cc was not 
compiled.

So, in my opinion, there are some #ifdef HAVE_POOL_OF_THREADS  lines that are 
missing in common threadpool code that would prevent to use threadpool generic 
routines.
However, I do not master this code.


Why not implement on AIX the generic pool of thread and thus add AIX to the 
above IF( ) test of sql/CMakeLists.txt ??
The reason if that, out of the 4 possibilities listed in 
sql/threadpool_generic.cc : epoll, KQUEUE, port, and IOCP, none of them are 
available on AIX with the EXACT required behavior. We have to implement 
something miming epoll(), we think. Before fixing this performance improvement, 
we'd like to be able to build & test MariaDB for all its other features.

sql/threadpool_generic.cc :
  #ifdef __linux__
    #include <sys/epoll.h>
    typedef struct epoll_event native_event;
  #elif defined(HAVE_KQUEUE)
    #include <sys/event.h>
    typedef struct kevent native_event;
  #elif defined (__sun)
    #include <port.h>
    typedef port_event_t native_event;
  #elif defined (HAVE_IOCP)
    typedef OVERLAPPED_ENTRY native_event;
  #else
  #error threadpool is not available on this platform          <<---- AIX
  #endif


Regards,

Tony


/opt/freeware/bin/g++
        -O2
        -g
        -L/opt/freeware/lib64
        -D_GNU_SOURCE
        -D_FILE_OFFSET_BITS=64
        -D_LARGEFILE_SOURCE
        -maix64
        -Wl,-bbigtoc
        -fPIC
        -O2
        -g
        -L/opt/freeware/lib64
        -D_GNU_SOURCE
        -D_FILE_OFFSET_BITS=64
        -D_LARGEFILE_SOURCE
        -maix64
        -Wl,-bbigtoc
        -fPIC
        -pie
        -fPIC
        -fno-rtti
        -O3
        -g
        -static-libgcc
        -fno-omit-frame-pointer
        -fno-strict-aliasing
        -Wno-uninitialized
        -D_FORTIFY_SOURCE=2
        -DDBUG_OFF
        -Wl,-blibpath:/opt/freeware/lib:/opt/freeware/lib64:/usr/lib:/lib
        -lpgport
        CMakeFiles/explain_filename-t.dir/explain_filename-t.cc.o
        -o
        explain_filename-t
        -Wl,-bnoipath
        
-Wl,-blibpath:/opt/freeware/lib64:/opt/freeware/lib/gcc/powerpc-ibm-aix7.2.0.0/8.2.0/ppc64:/opt/freeware/lib/ppc64:/opt/freeware/lib/gcc/powerpc-ibm-aix7.2.0.0/8.2.0:/opt/freeware/lib:/usr/lib:/lib
        -lpthreads
        ../../sql/libsql.a
        ../mytap/libmytap.a
        ../../storage/archive/libarchive.a
        ../../storage/blackhole/libblackhole.a
        ../../storage/csv/libcsv.a
        ../../storage/federatedx/libfederatedx.a
        ../../storage/heap/libheap.a
        ../../storage/innobase/libinnobase.a
        -llz4
        ../../storage/maria/libaria.a
        ../../storage/myisam/libmyisam.a
        ../../storage/myisammrg/libmyisammrg.a
        ../../storage/sequence/libsequence.a
        ../../plugin/feedback/libfeedback.a
        ../../plugin/user_variables/libuser_variables.a
        ../../plugin/userstat/libuserstat.a
        ../../sql/libpartition.a
        ../../sql/libsql_sequence.a
        ../../mysys/libmysys.a
        ../../mysys_ssl/libmysys_ssl.a
        ../../dbug/libdbug.a
        ../../mysys/libmysys.a
        ../../mysys_ssl/libmysys_ssl.a
        ../../dbug/libdbug.a
        /usr/lib64/libz.so
        -lm
        ../../extra/yassl/libyassl.a
        ../../extra/yassl/taocrypt/libtaocrypt.a
        ../../strings/libstrings.a
        ../../vio/libvio.a
        ../../pcre/libpcre.a
        ../../wsrep-lib/src/libwsrep-lib.a
        -lpthread
        -ldl
        ../../wsrep-lib/wsrep-API/libwsrep_api_v26.a
        -lpthreads

_______________________________________________
Mailing list: 
https://launchpad.net/~maria-developers<https://eur01.safelinks.protection.outlook.com/?url=https:%2F%2Flaunchpad.net%2F~maria-developers&data=02%7C01%7Ctony.reix%40atos.net%7Cb5e9b806e428485b785c08d6c3e3e1b8%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C636911780844422005&sdata=fezpM4YW6k3ej74spDF4f2SoK21xVc9Utao4dj41HVc%3D&reserved=0>
Post to     : maria-developers@lists.launchpad.net
Unsubscribe : 
https://launchpad.net/~maria-developers<https://eur01.safelinks.protection.outlook.com/?url=https:%2F%2Flaunchpad.net%2F~maria-developers&data=02%7C01%7Ctony.reix%40atos.net%7Cb5e9b806e428485b785c08d6c3e3e1b8%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C636911780844422005&sdata=fezpM4YW6k3ej74spDF4f2SoK21xVc9Utao4dj41HVc%3D&reserved=0>
More help   : 
https://help.launchpad.net/ListHelp<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.launchpad.net%2FListHelp&data=02%7C01%7Ctony.reix%40atos.net%7Cb5e9b806e428485b785c08d6c3e3e1b8%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C636911780844432015&sdata=wnvMOIHEaCVxIiDivdPl8M81OP2dmkbpIvjC3T5SmZo%3D&reserved=0>



_______________________________________________
Mailing list: https://launchpad.net/~maria-developers
Post to     : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Reply via email to