Re: [pcre-dev] PCRE2 COPYING file is the GPL3 license

[ph10]

On Tue, 5 Jan 2016, Giuseppe D'Angelo wrote:

> I guess it's a benign mistake? LICENCE says that PCRE2 is BSD. :)

Oh, gosh, how did that get there? The PCRE1 COPYING file just contains 
this:

  Please see the file LICENCE in the PCRE distribution for licensing 
  details.
  
The PCRE2 file should be the same (with PCRE=>PCRE2). I will fix it
before the 10.21 release. In fact, I have done it right now. Thanks for
noticing.

Philip

-- 
Philip Hazel

-- 
## List details at https://lists.exim.org/mailman/listinfo/pcre-dev 

Re: [pcre-dev] PCRE2 and thread safety of jit compilation?

[Zoltán Herczeg]

>The biggest problem is having this working (and claim as supported)
>amongst the huge number of compilers and platforms that PCRE2 runs on.
>That's not an easy task.

this is exactly my problem :)

Perhaps we could start by supporting some platforms, and gradually cover more 
with the community help. I heard that asm volatile forces GCC (and perhaps 
clang) to disable moving instructions around such asm blocks.

E.g:

statement1;
asm volatile (" ");
statement2;

Is it true, that statement1 is fully completed before statement2 is executed 
even if the assembly part is nothing?

Hence:

ptr->member = something;
asm volatile (" ");
result_ptr->member = ptr;

is executed in-order?

The CPU can still reorder stores. An x86 CPU does not have (need) data write 
barrier instruction as far as I know. Recent ARM 32 CPUs has data write 
barrier. Could somebody tell me how can I test whether this instruction is 
available at compile time? ARM 64 should not be a problem. I have not checked 
other CPUs yet.

There is one more thing. This theoretically affects everything, not just JIT 
compilation. If we compile a pattern with pcre2_compile, it is possible that 
the result pointer has been shared with another thread, but the compiled 
pattern data is not.

Main thread:

compiled_pcre_pattern->byte_code = something;
return compiled_pcre_pattern;

shared_pattern = compiled_pcre_pattern;

Another thread:
match (shared_pattern, subject);

The byte_code part can be a garbage on the other thread (since it can be 
executed by another CPU). People did not complain about these effects before, 
is there a reason for that? I don't want to solve a non-existing problem.

Regards,
Zoltan


-- 
## List details at https://lists.exim.org/mailman/listinfo/pcre-dev 

Re: [pcre-dev] PCRE2 and thread safety of jit compilation?

[Giuseppe D'Angelo]

On Tue, Jan 5, 2016 at 5:12 PM, Zoltán Herczeg  wrote:
> Perhaps we could start by supporting some platforms, and gradually cover more 
> with the community help. I heard that asm volatile forces GCC (and perhaps 
> clang) to disable moving instructions around such asm blocks.
>
> E.g:
>
> statement1;
> asm volatile (" ");
> statement2;
>
> Is it true, that statement1 is fully completed before statement2 is executed 
> even if the assembly part is nothing?

I don't think so, it's even documented:
https://gcc.gnu.org/onlinedocs/gcc/Extended-Asm.html

"Note that the compiler can move even volatile asm instructions
relative to other code"

What you may use on older GCCs is __sync_synchronize() (full barrier),
assuming pointer assignments to be atomic.

Newer GCCs have __atomic_load / __atomic_store with the
__ATOMIC_ACQUIRE and __ATOMIC_RELEASE memory models; even newer GCCs
have C11, so the _Atomic type qualifier and  operations.

... totally open question marks about other platforms / compilers ...

> The CPU can still reorder stores. An x86 CPU does not have (need) data write 
> barrier instruction as far as I know. Recent ARM 32 CPUs has data write 
> barrier. Could somebody tell me how can I test whether this instruction is 
> available at compile time? ARM 64 should not be a problem. I have not checked 
> other CPUs yet.

I guess you want to check the exact ARM revision of the CPU? Some
detection code like this?

http://code.woboq.org/qt5/qtbase/src/corelib/global/qprocessordetection.h.html#90

> There is one more thing. This theoretically affects everything, not just JIT 
> compilation. If we compile a pattern with pcre2_compile, it is possible that 
> the result pointer has been shared with another thread, but the compiled 
> pattern data is not.
>
> Main thread:
>
> compiled_pcre_pattern->byte_code = something;
> return compiled_pcre_pattern;
>
> shared_pattern = compiled_pcre_pattern;
>
> Another thread:
> match (shared_pattern, subject);
>
> The byte_code part can be a garbage on the other thread (since it can be 
> executed by another CPU). People did not complain about these effects before, 
> is there a reason for that? I don't want to solve a non-existing problem.

Well, this is a problem of PCRE users and how they share that
"shared_pattern" across threads. The "proper" way is making it an
atomic pointer, then turning the assignment into an atomic
store/release and the read in match into an atomic load/acquire.

Cheers,
-- 
Giuseppe D'Angelo

-- 
## List details at https://lists.exim.org/mailman/listinfo/pcre-dev 

[pcre-dev] [Bug 1770] New: PCRE fails to detect a circular reference of some numbered patterns

[admin]

https://bugs.exim.org/show_bug.cgi?id=1770

Bug ID: 1770
   Summary: PCRE fails to detect a circular reference of some
numbered patterns
   Product: PCRE
   Version: 10.20 (PCRE2)
  Hardware: x86
OS: All
Status: NEW
  Severity: security
  Priority: medium
 Component: Code
  Assignee: p...@hermes.cam.ac.uk
  Reporter: v.bakai...@gmail.com
CC: pcre-dev@exim.org

PCRE fails to detect a circular reference in the following regex:

(((?3)))((?1))

This causes an infinite loop that eventually segfaults at pcre_compile.c:2338 

vbakaitis@fuzz:~$ pcre2grep "(((?3)))((?1))" .
Segmentation fault

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/pcre-dev 

Re: [pcre-dev] PCRE2 and thread safety of jit compilation?

[Giuseppe D'Angelo]

On Tue, Jan 5, 2016 at 8:22 AM, Zoltán Herczeg  wrote:
> I understand the basic principles, but I cannot judge the side effects and 
> limitations (CPU/OS/compiler) of those changes. It would be better to discuss 
> proposed patches and land the best solution.

The biggest problem is having this working (and claim as supported)
amongst the huge number of compilers and platforms that PCRE2 runs on.
That's not an easy task.

I guess it's better to ask users to use read/write locking around
matching and jit compiling. Or we could think of adding back APIs like
pcre_study -- having the jit data returned as a separate piece of
information...

Cheers,
-- 
Giuseppe D'Angelo

-- 
## List details at https://lists.exim.org/mailman/listinfo/pcre-dev 

[pcre-dev] PCRE2 COPYING file is the GPL3 license

[Giuseppe D'Angelo]

I guess it's a benign mistake? LICENCE says that PCRE2 is BSD. :)

Cheers,
-- 
Giuseppe D'Angelo

-- 
## List details at https://lists.exim.org/mailman/listinfo/pcre-dev