Re: [HACKERS] new function for tsquery creartion

2017-10-13 Thread Victor Drobny 

On 2017-10-13 16:37, Alexey Chernyshov wrote:

Hi all,
I am extending phrase operator  is such way that it will have <n,m>
syntax that means from n to m words, so I will use such syntax (<n,m>)
further. I found that a AROUND(N) b is exactly the same as a <-N,N> b
and it can be replaced while parsing. So, what do you think of such
idea? In this patch I have noticed some unobvious behavior.


Thank you for the interest and review!


# select to_tsvector('Hello, cat world!') @@ queryto_tsquery('cat
AROUND(1) cat') as match;
match
---
 t

cat AROUND(1) cat is the same is "cat <1> cat || cat <0> cat" and:

# select to_tsvector('Hello, cat world!') @@ to_tsquery('cat <0> cat');
 ?column?
---
 t

It seems to be a proper logic behavior but it is a possible pitfall,
maybe it should be documented?


It is a tricky question. I think that this interpretation is confusing, 
so

better to make it as <-N, -1> and <1, N>.


But more important question is how AROUND() operator should handle stop
words? Now it works as:

# select queryto_tsquery('cat <2> (a AROUND(10) rat)');
 queryto_tsquery
--
 'cat' <12> 'rat'
(1 row)

# select queryto_tsquery('cat <2> a AROUND(10) rat');
queryto_tsquery

 'cat' AROUND(12) 'rat'
(1 row)

In my opinion it should be like:
cat <2> (a AROUND(10) rat) == cat <2,2> (a <-10,10> rat) == cat <-8,12>
rat


I think that correct version is:
cat <2> (a AROUND(10) rat) == cat <2,2> (a <-10,10> rat) == cat <-2,12> 
rat.



cat <2> a AROUND(10) rat == cat <2,2> a <-10,10> rat = cat <-8, 12>
rat


It is a problem indeed. I did not catch it during implementation. Thank 
you

for pointing it out.


Now <n,m> operator can be replaced with combination of phrase
operator , AROUND(), and logical operators, but with <n,m> operator
it will be much painless. Correct me, please, if I am wrong.


I think that <n,m> operator is more general than around(n) so the last 
one
should be based on yours. However, i think, that taking negative 
parameters
is not the best idea because it is confusing. On top of that it is not 
so

necessary and i think it won`t be popular among users.
It seems to me that AROUND operator can be easily implemented with 
<n,m>,

also, it helps to avoid problems, that you showed above.

--
Victor Drobny
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] new function for tsquery creartion

2017-09-13 Thread Victor Drobny 

On 2017-09-09 06:03, Thomas Munro wrote:

Please send a rebased version of the patch for people to review and
test as that one has bit-rotted.

Hello,
Thank you for interest. In the attachment you can find rebased
version(based on 69835bc8988812c960f4ed5aeee86b62ac73602a commit)
--
Victor Drobny
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Companydiff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 641b3b8..a694801 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -9523,6 +9523,18 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple

 
  
+  queryto_tsquery
+
+ queryto_tsquery( config regconfig ,  query text)
+
+tsquery
+produce tsquery from google like query
+queryto_tsquery('english', 'The Fat Rats')
+'fat'  'rat'
+   
+   
+
+ 
   querytree
  
  querytree(query tsquery)
diff --git a/doc/src/sgml/textsearch.sgml b/doc/src/sgml/textsearch.sgml
index fe630a6..999e4ad 100644
--- a/doc/src/sgml/textsearch.sgml
+++ b/doc/src/sgml/textsearch.sgml
@@ -797,13 +797,15 @@ UPDATE tt SET ti =

 PostgreSQL provides the
 functions to_tsquery,
-plainto_tsquery, and
-phraseto_tsquery
+plainto_tsquery,
+phraseto_tsquery and
+queryto_tsquery
 for converting a query to the tsquery data type.
 to_tsquery offers access to more features
 than either plainto_tsquery or
 phraseto_tsquery, but it is less forgiving
-about its input.
+about its input. queryto_tsquery provides a 
+different, Google like syntax to create tsquery.

 

@@ -960,8 +962,68 @@ SELECT phraseto_tsquery('english', 'The Fat  Rats:C');
 -
  'fat' - 'rat' - 'c'
 
+
+
+
+queryto_tsquery( config regconfig,  querytext text) returns tsquery
+
+
+   
+queryto_tsquery creates a tsquery from a unformated text.
+But instead of plainto_tsquery and phraseto_tsquery it won't
+ignore already placed operations. This function supports following operators:
+
+ 
+  
+   '"some text" - any text inside quote signs will be treated as a phrase and will be
+performed like in phraseto_tsquery.
+  
+ 
+ 
+  
+   'OR' - standard logical operator. It is just an alias for '|'' sign.
+  
+ 
+ 
+  
+   'terma AROUND(N) termb' - this operation will match if the distance between 
+   terma and termb is less than N.
+  
+ 
+ 
+  
+   '-' - standard logical negation sign. It is an alias for '!' sign.
+  
+ 
+
+Other missing operators will be replaced by AND like in plainto_tsquery.

 
+   
+Examples:
+
+  select queryto_tsquery('The fat rats');
+   queryto_tsquery 
+  -
+   'fat' & 'rat'
+  (1 row)
+
+
+  select queryto_tsquery('"supernovae stars" AND -crab');
+ queryto_tsquery  
+  --
+   'supernova' <-> 'star' & !'crab'
+(1 row)
+
+
+  select queryto_tsquery('-run AROUND(5) "gnu debugger" OR "I like bananas"');
+queryto_tsquery  
+  ---
+   !'run' AROUND(5) 'gnu' <-> 'debugg' | 'like' <-> 'banana'
+  (1 row)
+
+
+
   
 
   
diff --git a/src/backend/tsearch/to_tsany.c b/src/backend/tsearch/to_tsany.c
index 35d9ab2..e820042 100644
--- a/src/backend/tsearch/to_tsany.c
+++ b/src/backend/tsearch/to_tsany.c
@@ -390,7 +390,8 @@ add_to_tsvector(void *_state, char *elem_value, int elem_len)
  * and different variants are ORed together.
  */
 static void
-pushval_morph(Datum opaque, TSQueryParserState state, char *strval, int lenval, int16 weight, bool prefix)
+pushval_morph(Datum opaque, TSQueryParserState state, char *strval, int lenval,
+			int16 weight, bool prefix, bool isphrase)
 {
 	int32		count = 0;
 	ParsedText	prs;
@@ -423,7 +424,12 @@ pushval_morph(Datum opaque, TSQueryParserState state, char *strval, int lenval,
 	/* put placeholders for each missing stop word */
 	pushStop(state);
 	if (cntpos)
-		pushOperator(state, data->qoperator, 1);
+	{
+		if (isphrase)
+			pushOperator(state, OP_PHRASE, 1);
+		else
+			pushOperator(state, data->qoperator, 1);
+	}
 	cntpos++;
 	pos++;
 }
@@ -464,7 +470,10 @@ pushval_morph(Datum opaque, TSQueryParserState state, char *strval, int lenval,
 			if (cntpos)
 			{
 /* distance may be useful */
-pushOperator(state, data->qoperator, 1);
+if (isphrase)
+	pushOperator(state, OP_PHRASE, 1);
+else
+	pushOperator(state, data->qoperator, 1);
 			}
 
 			cntpos++;
@@ -490,6 +499,7 @@ to_tsquery_byid(PG_FUNC

Re: [HACKERS] Red-Black tree traversal tests

2017-09-08 Thread Victor Drobny 

On 2017-09-08 15:23, Thomas Munro wrote:
On Fri, Sep 8, 2017 at 9:03 PM, Victor Drobny <v.dro...@postgrespro.ru> 
wrote:
Thank you very much for your review. In the attachment you can find v2 
of

the patch.


FYI this version crashes for me:

test test_rbtree  ... FAILED (test process exited with exit 
code 2)


It's trying to call rb->combiner which is null.


Thank you for pointing on it. Here is a fixed version.

--
Victor Drobny
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Companydiff --git a/src/test/modules/Makefile b/src/test/modules/Makefile
index 3ce9904..b7ed0af 100644
--- a/src/test/modules/Makefile
+++ b/src/test/modules/Makefile
@@ -13,6 +13,7 @@ SUBDIRS = \
 		  test_extensions \
 		  test_parser \
 		  test_pg_dump \
+		  test_rbtree \
 		  test_rls_hooks \
 		  test_shm_mq \
 		  worker_spi
diff --git a/src/test/modules/test_rbtree/Makefile b/src/test/modules/test_rbtree/Makefile
new file mode 100644
index 000..4e53e86
--- /dev/null
+++ b/src/test/modules/test_rbtree/Makefile
@@ -0,0 +1,21 @@
+# src/test/modules/test_rbtree/Makefile
+
+MODULE_big = test_rbtree
+OBJS = test.o $(WIN32RES)
+PGFILEDESC = "test_rbtree - rbtree traversal testing"
+
+EXTENSION = test_rbtree
+DATA = test_rbtree--1.0.sql
+
+REGRESS = test_rbtree
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/test_rbtree
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/test_rbtree/README b/src/test/modules/test_rbtree/README
new file mode 100644
index 000..40f586d
--- /dev/null
+++ b/src/test/modules/test_rbtree/README
@@ -0,0 +1,20 @@
+test_rbtree is a module tests for checking the correctness of all kinds of
+traversal of red-black tree. Right now rbtree in postgres has 4 kinds of
+traversals: Left-Current-Right, Right-Current-Left, Current-Left-Right and
+Left-Right-Current.
+
+This extension has 4 functions. Each function checks one traversal.
+The checking the correctness of first two types are based on the fact that
+red-black tree is a binary search tree, so the elements should be iterated in
+increasing(for Left-Current-Right) or decreasing(for Right-Current-Left)
+order.
+In order to verify last two strategies, we will check the sequence if it is
+correct or not. For given pre- or post- order traversing of binary search tree
+it is always possible to say is it correct or not and to rebuild original tree.
+The idea is based on the fact that in such traversal sequence is always
+possible to determine current node, left subtree and right subtree.
+
+Also, this module is checking the correctness of the find, delete and leftmost
+operation.
+
+These tests are performed on red-black trees that store integers.
diff --git a/src/test/modules/test_rbtree/expected/test_rbtree.out b/src/test/modules/test_rbtree/expected/test_rbtree.out
new file mode 100644
index 000..8223e81
--- /dev/null
+++ b/src/test/modules/test_rbtree/expected/test_rbtree.out
@@ -0,0 +1,7 @@
+CREATE EXTENSION test_rbtree;
+SELECT testrbtree(10);
+ testrbtree 
+
+ 
+(1 row)
+
diff --git a/src/test/modules/test_rbtree/int_rbtree.h b/src/test/modules/test_rbtree/int_rbtree.h
new file mode 100644
index 000..57754bf
--- /dev/null
+++ b/src/test/modules/test_rbtree/int_rbtree.h
@@ -0,0 +1,49 @@
+/*--
+ *
+ * int_rbtree.h
+ *		Definitions for integer red-black tree
+ *
+ * Copyright (c) 2013-2017, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		src/test/modules/test_rbtree/int_rbtree.h
+ *
+ * -
+ */
+
+#ifndef INT_RBTREE_H
+#define INT_RBTREE_H
+
+#include "lib/rbtree.h"
+
+typedef struct IntRBTreeNode
+{
+	RBNode		rbnode;
+	int			key;
+
+} IntRBTreeNode;
+
+static int
+cmp(const RBNode *a, const RBNode *b, void *arg)
+{
+	const IntRBTreeNode *ea = (const IntRBTreeNode *) a;
+	const IntRBTreeNode *eb = (const IntRBTreeNode *) b;
+
+	return ea->key - eb->key;
+}
+
+static RBNode *
+alloc(void *arg)
+{
+	IntRBTreeNode *ea;
+	ea = malloc(sizeof(IntRBTreeNode));
+	return (RBNode *) ea;
+}
+
+static void
+fr(RBNode * node, void *arg)
+{
+	free(node);
+}
+
+#endif /* INT_RBTREE_H */
diff --git a/src/test/modules/test_rbtree/sql/test_rbtree.sql b/src/test/modules/test_rbtree/sql/test_rbtree.sql
new file mode 100644
index 000..8263df3
--- /dev/null
+++ b/src/test/modules/test_rbtree/sql/test_rbtree.sql
@@ -0,0 +1,3 @@
+CREATE EXTENSION test_rbtree;
+
+SELECT testrbtree(10);
diff --git a/src/test/modules/test_rbtree/test.c b/src/test/modules/test_rbtree/test.c
new file mode 100644
index 000..e47e637
--- /dev/null
+++ b/src/test/module

Re: [HACKERS] Red-Black tree traversal tests

2017-09-08 Thread Victor Drobny 

Dear Tom,

Thank you very much for your review. In the attachment you can find v2 
of the patch.


On 2017-09-07 01:38, Tom Lane wrote:

[ btw, please don't cc pgsql-hackers-owner, the list moderators don't
need the noise ]

Aleksander Alekseev <a.aleks...@postgrespro.ru> writes:

I would say that this patch is in a pretty good shape now. And I see a
99% code coverage of rbtree.c. Let's see what committers think.


I took a quick look through the patch --- haven't tried to compile it
or anything like that --- and have a few comments:

* There's some typos, eg extention should be extension, triversal
should be traversal.  Maybe try a spell checker?


Done. I fixed all spelling mistakes that i found.



* The diff complains about lack of file-ending newlines in several
places

* There's something weird at the start of test.c:

@@ -0,0 +1,577 @@
+/*--

Maybe your compiler thinks that's a BOM?  It's hard to see how it
compiles otherwise.


Now it is in UTF-8 without BOM. It seems that there is no such data in 
the beginning

of the test.c


* I think it might be simpler to have the module expose just one SQL
function that invokes all these individual tests internally.  Less
boilerplate text that way, and less to change if you add more tests
later.  Also, you might consider passing in TEST_SIZE as an argument
of the SQL function instead of having it be hard-wired.

You are right. Done.


* We don't do C++-style (//) comments around here.  Please use C
style.  (You might consider running the code through pgindent,
which will convert those comments automatically.)


Fixed.



* It's also generally not per project style to use malloc/calloc/free
directly in backend code; and it's certainly not cool to use malloc or
calloc and then not check for a null result.  Use palloc instead.  
Given

the short runtime of this test, you likely needn't be too worried about
pfree'ing stuff.

* _PG_init() declaration seems to be a leftover?   doesn't
belong here either, as postgres.h will bring that in for you.

* I know next to zip about red-black trees, but it disturbs me that
the traversal tests use trees whose values were inserted in strictly
increasing order.  Seems like that's not proving as much as it could.
I wonder how hard it would be to insert those integers in random order.

* I'm not too pleased that the rb_find calls mostly just assume that
the find won't return NULL.  You should be testing for that and 
reporting

the problem, not just dying on a null pointer crash if it happens.


Done.



* Possibly the tests should exercise rb_delete on keys *not* present.
And how about insertion of already-existing keys?  Although that's
a bit outside the scope of testing traversal, so if you want to leave
it for some future expansion, that'd be OK.


Deletion requires to get pointer to the tree node. Otherwise it could 
break

the tree. It is mentioned in the description of the rb_delete function.
" * "node" must have previously been found via rb_find or rb_leftmost."

Insertion of the same elements is managed by the specific implementation
of the tree. One of the input arguments of the rb_create function is
combiner function that will be called in case of repeated insertion.

However, during looking through this i found that nobody checks that
combiner function(as well as comparator, freefunc and allocfunc) is
not NULL. So if it was not specified, postgres will fall. I think that
it is better to add this checks.



I'll set this back to Waiting on Author.  I do encourage you to finish
it up.

    regards, tom lane


--
Victor Drobny
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Companydiff --git a/src/test/modules/Makefile b/src/test/modules/Makefile
index 3ce9904..b7ed0af 100644
--- a/src/test/modules/Makefile
+++ b/src/test/modules/Makefile
@@ -13,6 +13,7 @@ SUBDIRS = \
 		  test_extensions \
 		  test_parser \
 		  test_pg_dump \
+		  test_rbtree \
 		  test_rls_hooks \
 		  test_shm_mq \
 		  worker_spi
diff --git a/src/test/modules/test_rbtree/Makefile b/src/test/modules/test_rbtree/Makefile
new file mode 100644
index 000..4e53e86
--- /dev/null
+++ b/src/test/modules/test_rbtree/Makefile
@@ -0,0 +1,21 @@
+# src/test/modules/test_rbtree/Makefile
+
+MODULE_big = test_rbtree
+OBJS = test.o $(WIN32RES)
+PGFILEDESC = "test_rbtree - rbtree traversal testing"
+
+EXTENSION = test_rbtree
+DATA = test_rbtree--1.0.sql
+
+REGRESS = test_rbtree
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/test_rbtree
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/test_rbtree/README b/src/test/modules/test_rbtree/README
new file mode 100644
index 000..40f586d
--- /dev/null
+++ b/src/test/modu

Re: [HACKERS] Red-Black tree traversal tests

2017-08-02 Thread Victor Drobny 

I forgot to attach the patch. Sorry.
Here it is.
--
--
Victor Drobny
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Companydiff --git a/src/test/modules/Makefile b/src/test/modules/Makefile
index 3ce9904..b7ed0af 100644
--- a/src/test/modules/Makefile
+++ b/src/test/modules/Makefile
@@ -13,6 +13,7 @@ SUBDIRS = \
 		  test_extensions \
 		  test_parser \
 		  test_pg_dump \
+		  test_rbtree \
 		  test_rls_hooks \
 		  test_shm_mq \
 		  worker_spi
diff --git a/src/test/modules/test_rbtree/.gitignore b/src/test/modules/test_rbtree/.gitignore
new file mode 100644
index 000..5dcb3ff
--- /dev/null
+++ b/src/test/modules/test_rbtree/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/test_rbtree/Makefile b/src/test/modules/test_rbtree/Makefile
new file mode 100644
index 000..11a10cf
--- /dev/null
+++ b/src/test/modules/test_rbtree/Makefile
@@ -0,0 +1,21 @@
+# src/test/modules/test_rbtree/Makefile
+
+MODULE_big = test_rbtree
+OBJS = test.o $(WIN32RES)
+PGFILEDESC = "test_rbtree - rbtree triversal testing"
+
+EXTENSION = test_rbtree
+DATA = test_rbtree--1.0.sql
+
+REGRESS = test_rbtree
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/test_rbtree
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/test_rbtree/README b/src/test/modules/test_rbtree/README
new file mode 100644
index 000..8f4287e
--- /dev/null
+++ b/src/test/modules/test_rbtree/README
@@ -0,0 +1,20 @@
+test_rbtree is a module tests for checking the correctness of all kinds of
+traversal of red-black tree. Right now rbtree in postgres has 4 kinds of
+traversals: Left-Current-Right, Right-Current-Left, Current-Left-Right and
+Left-Right-Current.
+
+This extention has 4 functions. Each function checks one traversal.
+The checking the correctness of first two types are based on the fact that
+red-black tree is a binary search tree, so the elements should be iterated in
+increasing(for Left-Current-Right) or decreasing(for Right-Current-Left)
+order.
+In order to verify last two strategies, we will check the sequence if it is
+correct or not. For given pre- or post- order traversing of binary search tree
+it is always possible to say is it correct or not and to rebuild original tree.
+The idea is based on the fact that in such traversal sequence is always
+possible to determine current node, left subtree and right subtree.
+
+Also, this module is checking the correctness of the find, delete and leftmost
+operation.
+
+These tests are performed on red-black trees that store integers.
\ No newline at end of file
diff --git a/src/test/modules/test_rbtree/expected/test_rbtree.out b/src/test/modules/test_rbtree/expected/test_rbtree.out
new file mode 100644
index 000..cd4435b
--- /dev/null
+++ b/src/test/modules/test_rbtree/expected/test_rbtree.out
@@ -0,0 +1,43 @@
+CREATE EXTENSION test_rbtree;
+SELECT testleftright();
+ testleftright 
+---
+ 
+(1 row)
+
+SELECT testrightleft();
+ testrightleft 
+---
+ 
+(1 row)
+
+SELECT testdirect();
+ testdirect 
+
+ 
+(1 row)
+
+SELECT testinverted();
+ testinverted 
+--
+ 
+(1 row)
+
+SELECT testfind();
+ testfind 
+--
+ 
+(1 row)
+
+SELECT testleftmost();
+ testleftmost 
+--
+ 
+(1 row)
+
+SELECT testdelete();
+ testdelete 
+
+ 
+(1 row)
+
diff --git a/src/test/modules/test_rbtree/int_rbtree.h b/src/test/modules/test_rbtree/int_rbtree.h
new file mode 100644
index 000..d153616
--- /dev/null
+++ b/src/test/modules/test_rbtree/int_rbtree.h
@@ -0,0 +1,49 @@
+/*--
+ *
+ * int_rbtree.h
+ *		Definitions for integer red-black tree
+ *
+ * Copyright (c) 2013-2017, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		src/test/modules/test_rbtree/int_rbtree.h
+ *
+ * -
+ */
+
+#ifndef INT_RBTREE_H
+#define INT_RBTREE_H
+
+#include "lib/rbtree.h"
+
+typedef struct IntRBTreeNode
+{
+	RBNode		rbnode;
+	int			key;
+
+} IntRBTreeNode;
+
+static int
+cmp(const RBNode *a, const RBNode *b, void *arg)
+{
+	const IntRBTreeNode *ea = (const IntRBTreeNode *) a;
+	const IntRBTreeNode *eb = (const IntRBTreeNode *) b;
+
+	return ea->key - eb->key;
+}
+
+static RBNode *
+alloc(void *arg)
+{
+	IntRBTreeNode *ea;
+	ea = malloc(sizeof(IntRBTreeNode));
+	return (RBNode *) ea;
+}
+
+static void
+fr(RBNode * node, void *arg)
+{
+	free(node);
+}
+
+#endif // INT_RBTREE_H
diff --git a/src/test/modules/test_rbtree/sql/test_rbtree.sql b/src/test/modules/test_rbtree/sql/test_rbtree.sql
new file mode 100644
index 000..3bedff2
--- /dev/null
+++ b/src/test/modules/test_rbtree/sql/test_rbtree.sql
@@ -0,0

Re: [HACKERS] Red-Black tree traversal tests

2017-08-02 Thread Victor Drobny 

Hello,

Thank you for the reviewing.
If it's not too much trouble perhaps you could write a few more test 
so
we would have 100% test coverage for rbtree, could modify it safely 
and

be sure that it actually works when someone will need the rest of its
functionality?


Done. Now all of the functions in rbtree.c are covered.


Also I would recommend to add your patch to the nearest commitfest [1].
Otherwise there is a good chance that everyone will forget about it
quite soon.

[1]: https://commitfest.postgresql.org/14/


Done. Here is the link: https://commitfest.postgresql.org/14/1225/

Thank you for attention!

--
--
Victor Drobny
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Red-Black tree traversal tests

2017-07-28 Thread Victor Drobny 

Hello,

Postgres now has its own red-black tree implementation. This tree has 4 
types of traversals. In the attachment, you can find module test that 
checks the correctness of tree traversal strategies.


I hope that someone can find it useful.

Thank you for attention!

--
--
Victor Drobny
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Companydiff --git a/src/test/modules/Makefile b/src/test/modules/Makefile
index 3ce9904..b7ed0af 100644
--- a/src/test/modules/Makefile
+++ b/src/test/modules/Makefile
@@ -13,6 +13,7 @@ SUBDIRS = \
 		  test_extensions \
 		  test_parser \
 		  test_pg_dump \
+		  test_rbtree \
 		  test_rls_hooks \
 		  test_shm_mq \
 		  worker_spi
diff --git a/src/test/modules/test_rbtree/.gitignore b/src/test/modules/test_rbtree/.gitignore
new file mode 100644
index 000..5dcb3ff
--- /dev/null
+++ b/src/test/modules/test_rbtree/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/test_rbtree/Makefile b/src/test/modules/test_rbtree/Makefile
new file mode 100644
index 000..11a10cf
--- /dev/null
+++ b/src/test/modules/test_rbtree/Makefile
@@ -0,0 +1,21 @@
+# src/test/modules/test_rbtree/Makefile
+
+MODULE_big = test_rbtree
+OBJS = test.o $(WIN32RES)
+PGFILEDESC = "test_rbtree - rbtree triversal testing"
+
+EXTENSION = test_rbtree
+DATA = test_rbtree--1.0.sql
+
+REGRESS = test_rbtree
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/test_rbtree
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/test_rbtree/README b/src/test/modules/test_rbtree/README
new file mode 100644
index 000..9e6c977
--- /dev/null
+++ b/src/test/modules/test_rbtree/README
@@ -0,0 +1,17 @@
+test_rbtree is a module tests for checking the correctness of all kinds of
+traversal of red-black tree. Right now rbtree in postgres has 4 kinds of
+traversals: Left-Current-Right, Right-Current-Left, Current-Left-Right and
+Left-Right-Current.
+
+This extention has 4 functions. Each function checks one traversal.
+The checking the correctness of first two types are based on the fact that
+red-black tree is a binary search tree, so the elements should be iterated in
+increasing(for Left-Current-Right) or decreasing(for Right-Current-Left)
+order.
+In order to verify last two strategies, we will check the sequence if it is
+correct or not. For given pre- or post- order traversing of binary search tree
+it is always possible to say is it correct or not and to rebuild original tree.
+The idea is based on the fact that in such traversal sequence is always
+possible to determine current node, left subtree and right subtree.
+
+This tests are performed on red-black trees that store integers.
\ No newline at end of file
diff --git a/src/test/modules/test_rbtree/expected/test_rbtree.out b/src/test/modules/test_rbtree/expected/test_rbtree.out
new file mode 100644
index 000..be8cd75
--- /dev/null
+++ b/src/test/modules/test_rbtree/expected/test_rbtree.out
@@ -0,0 +1,25 @@
+CREATE EXTENSION test_rbtree;
+SELECT testleftright();
+ testleftright 
+---
+ 
+(1 row)
+
+SELECT testrightleft();
+ testrightleft 
+---
+ 
+(1 row)
+
+SELECT testdirect();
+ testdirect 
+
+ 
+(1 row)
+
+SELECT testinverted();
+ testinverted 
+--
+ 
+(1 row)
+
diff --git a/src/test/modules/test_rbtree/int_rbtree.h b/src/test/modules/test_rbtree/int_rbtree.h
new file mode 100644
index 000..d153616
--- /dev/null
+++ b/src/test/modules/test_rbtree/int_rbtree.h
@@ -0,0 +1,49 @@
+/*--
+ *
+ * int_rbtree.h
+ *		Definitions for integer red-black tree
+ *
+ * Copyright (c) 2013-2017, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ *		src/test/modules/test_rbtree/int_rbtree.h
+ *
+ * -
+ */
+
+#ifndef INT_RBTREE_H
+#define INT_RBTREE_H
+
+#include "lib/rbtree.h"
+
+typedef struct IntRBTreeNode
+{
+	RBNode		rbnode;
+	int			key;
+
+} IntRBTreeNode;
+
+static int
+cmp(const RBNode *a, const RBNode *b, void *arg)
+{
+	const IntRBTreeNode *ea = (const IntRBTreeNode *) a;
+	const IntRBTreeNode *eb = (const IntRBTreeNode *) b;
+
+	return ea->key - eb->key;
+}
+
+static RBNode *
+alloc(void *arg)
+{
+	IntRBTreeNode *ea;
+	ea = malloc(sizeof(IntRBTreeNode));
+	return (RBNode *) ea;
+}
+
+static void
+fr(RBNode * node, void *arg)
+{
+	free(node);
+}
+
+#endif // INT_RBTREE_H
diff --git a/src/test/modules/test_rbtree/sql/test_rbtree.sql b/src/test/modules/test_rbtree/sql/test_rbtree.sql
new file mode 100644
index 000..edd90c1
--- /dev/null
+++ b/src/test/modules/test_rbtree/sql/test_rbtree.sql
@@ -0,0 +1,6 @@
+CREATE EXTENSION test_rbtree;
+
+SELECT testleftrigh

Re: [HACKERS] pg_dump issues

2017-07-25 Thread Victor Drobny 

We can't create any schema dump with another (user defined) name. E.g.
we dump schema test and we want to save it's dump with test2 name in
any format. Those refers to databases dump.


Hello,

Do you expect to have some flag like '--rename=test->test2'?
Will dump with test replaced by test2(of course only in related places) 
be valid dump in this case?
What is the possible scenario for the renaming option? Is it doing to be 
dumping of the one schema only?
Or it could be dump of database? In this case pg_dump should also 
support multiple rules for renaming.


Thank you for attention!

--
--
Victor Drobny
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] new function for tsquery creartion

2017-07-19 Thread Victor Drobny 

Dear all,

Now Postgres has a few functions to create tsqueries for full text 
search. The main one is the to_tsquery function that allows to make 
query with any operation. But to make correct query all of the operators 
should be specified explicitly. In order to make it easier postgres has 
functions like plainto_tsquery and phraseto_tsquery which allow to make 
tsqueries from strings. But they are not flexible enough.


Let me introduce new function for full text search query creation(which 
is called 'queryto_tsquery'). It takes 'google like' query string and 
translates it to tsquery.

The main features are the following:
All the text inside double quotes would be treated as a phrase("a b c" 
-> 'a <-> b  <-> c')
New operator AROUND(N). It matches if the distance between words(or 
maybe phrases) is less than or equal to N.

Alias for !('-rat' is the same as '!rat')
Alias for |('dog OR cat' is the same as 'dog | cat')

As a plainto_tsquery and phraseto_tsquery it will fill operators by 
itself, but already placed operations won't be ignored. It allows to 
combine two approaches.


In the attachment you can find patch with the new features, tests and 
documentation for it.

What do you think about it?

Thank you very much for the attention!

--
--
Victor Drobny
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Companydiff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index e073f7b..d6fb4ce 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -9494,6 +9494,18 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple

 
  
+  queryto_tsquery
+
+ queryto_tsquery( config regconfig ,  query text)
+
+tsquery
+produce tsquery from google like query
+queryto_tsquery('english', 'The Fat Rats')
+'fat'  'rat'
+   
+   
+
+ 
   querytree
  
  querytree(query tsquery)
diff --git a/doc/src/sgml/textsearch.sgml b/doc/src/sgml/textsearch.sgml
index fe630a6..999e4ad 100644
--- a/doc/src/sgml/textsearch.sgml
+++ b/doc/src/sgml/textsearch.sgml
@@ -797,13 +797,15 @@ UPDATE tt SET ti =

 PostgreSQL provides the
 functions to_tsquery,
-plainto_tsquery, and
-phraseto_tsquery
+plainto_tsquery,
+phraseto_tsquery and
+queryto_tsquery
 for converting a query to the tsquery data type.
 to_tsquery offers access to more features
 than either plainto_tsquery or
 phraseto_tsquery, but it is less forgiving
-about its input.
+about its input. queryto_tsquery provides a 
+different, Google like syntax to create tsquery.

 

@@ -960,8 +962,68 @@ SELECT phraseto_tsquery('english', 'The Fat  Rats:C');
 -
  'fat' - 'rat' - 'c'
 
+
+
+
+queryto_tsquery( config regconfig,  querytext text) returns tsquery
+
+
+   
+queryto_tsquery creates a tsquery from a unformated text.
+But instead of plainto_tsquery and phraseto_tsquery it won't
+ignore already placed operations. This function supports following operators:
+
+ 
+  
+   '"some text" - any text inside quote signs will be treated as a phrase and will be
+performed like in phraseto_tsquery.
+  
+ 
+ 
+  
+   'OR' - standard logical operator. It is just an alias for '|'' sign.
+  
+ 
+ 
+  
+   'terma AROUND(N) termb' - this operation will match if the distance between 
+   terma and termb is less than N.
+  
+ 
+ 
+  
+   '-' - standard logical negation sign. It is an alias for '!' sign.
+  
+ 
+
+Other missing operators will be replaced by AND like in plainto_tsquery.

 
+   
+Examples:
+
+  select queryto_tsquery('The fat rats');
+   queryto_tsquery 
+  -
+   'fat' & 'rat'
+  (1 row)
+
+
+  select queryto_tsquery('"supernovae stars" AND -crab');
+ queryto_tsquery  
+  --
+   'supernova' <-> 'star' & !'crab'
+(1 row)
+
+
+  select queryto_tsquery('-run AROUND(5) "gnu debugger" OR "I like bananas"');
+queryto_tsquery  
+  ---
+   !'run' AROUND(5) 'gnu' <-> 'debugg' | 'like' <-> 'banana'
+  (1 row)
+
+
+
   
 
   
diff --git a/src/backend/tsearch/to_tsany.c b/src/backend/tsearch/to_tsany.c
index 18368d1..10fd8c3 100644
--- a/src/backend/tsearch/to_tsany.c
+++ b/src/backend/tsearch/to_tsany.c
@@ -414,7 +414,8 @@ add_to_tsvector(void *_state, char *elem_value, int elem_len)
  * and different variants are ORed together.
  */
 static void
-pushval_morph(Datum opaque, TSQueryParserState state, char *strval, int lenval, int16 weight, b

Re: [HACKERS] merge psql ef/ev sf/sv handling functions

2017-07-18 Thread Victor Drobny 

On 2017-03-31 21:04, Fabien COELHO wrote:

Hello,

While reviewing Corey's \if patch, I complained that there was some
amount of copy-paste in "psql/command.c".

Here is an attempt at merging some functions which removes 160 lines of 
code.


Hello,

I was looking through your patch. It seems good, the of the functions 
was very similar.
I have a question for you. What was the reason to replace 
"printfPQExpBuffer" by "resetPQExpBuffer" and "appendPQExpBufferStr"?


Thank you for attention!

--
--
Victor Drobny
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Authentification method on client side checking

2017-07-09 Thread Victor Drobny 

Hello,

Despite the addition of SCRAM authentification to PostgreSQL 10, MITM 
attack can be performed by saying that the server supports, for example, 
only md5 authentication. The possible solution for it is checking 
authentification method on a client side and reject connections that 
could be unsafe.


Postgresql server can require unencrypted password passing, md5, scram, 
gss or sspi authentification.


In the attached patch you can find the solution for it. The new provided 
features are the following:
The parameter with acceptable authentification methods can be passed 
into connection methods of libpq library.
Also, this parameter can be specified to psql as a command line 
argument.
The documentation for command line arguments of psql and arguments of 
libpq methods are also presented.


Thank you for attention!

Best,
--
--
Victor Drobny
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Companydiff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 8068a28..1877b2d 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -75,6 +75,7 @@ struct adhoc_opts
 	bool		no_psqlrc;
 	bool		single_txn;
 	bool		list_dbs;
+	char	   *authtype;
 	SimpleActionList actions;
 };
 
@@ -213,7 +214,7 @@ main(int argc, char *argv[])
 	/* loop until we have a password if requested by backend */
 	do
 	{
-#define PARAMS_ARRAY_SIZE	8
+#define PARAMS_ARRAY_SIZE	9
 		const char **keywords = pg_malloc(PARAMS_ARRAY_SIZE * sizeof(*keywords));
 		const char **values = pg_malloc(PARAMS_ARRAY_SIZE * sizeof(*values));
 
@@ -232,8 +233,10 @@ main(int argc, char *argv[])
 		values[5] = pset.progname;
 		keywords[6] = "client_encoding";
 		values[6] = (pset.notty || getenv("PGCLIENTENCODING")) ? NULL : "auto";
-		keywords[7] = NULL;
-		values[7] = NULL;
+		keywords[7] = "acc_auth";
+		values[7] = options.authtype;
+		keywords[8] = NULL;
+		values[8] = NULL;
 
 		new_pass = false;
 		pset.db = PQconnectdbParams(keywords, values, true);
@@ -441,6 +444,7 @@ parse_psql_options(int argc, char *argv[], struct adhoc_opts * options)
 		{"single-line", no_argument, NULL, 'S'},
 		{"tuples-only", no_argument, NULL, 't'},
 		{"table-attr", required_argument, NULL, 'T'},
+		{"authtype", required_argument, NULL, 'u'},
 		{"username", required_argument, NULL, 'U'},
 		{"set", required_argument, NULL, 'v'},
 		{"variable", required_argument, NULL, 'v'},
@@ -458,7 +462,7 @@ parse_psql_options(int argc, char *argv[], struct adhoc_opts * options)
 
 	memset(options, 0, sizeof *options);
 
-	while ((c = getopt_long(argc, argv, "aAbc:d:eEf:F:h:HlL:no:p:P:qR:sStT:U:v:VwWxXz?01",
+	while ((c = getopt_long(argc, argv, "aAbc:d:eEf:F:h:HlL:no:p:P:qR:sStT:u:U:v:VwWxXz?01",
 			long_options, )) != -1)
 	{
 		switch (c)
@@ -566,6 +570,9 @@ parse_psql_options(int argc, char *argv[], struct adhoc_opts * options)
 			case 'T':
 pset.popt.topt.tableAttr = pg_strdup(optarg);
 break;
+			case 'u':
+options->authtype = pg_strdup(optarg);
+break;
 			case 'U':
 options->username = pg_strdup(optarg);
 break;
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 4dc8924..b8e77d4 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -304,6 +304,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
 		"Target-Session-Attrs", "", 11, /* sizeof("read-write") = 11 */
 	offsetof(struct pg_conn, target_session_attrs)},
 
+	{"acc_auth", NULL, NULL, NULL,
+		"Acceptable authentification methods", "", 20,
+	offsetof(struct pg_conn, acc_auth)},
+
 	/* Terminating entry --- MUST BE LAST */
 	{NULL, NULL, NULL, NULL,
 	NULL, NULL, 0}
@@ -1000,6 +1004,43 @@ connectOptions2(PGconn *conn)
 		}
 	}
 
+	/* Validate acceptable authentification methods */
+	if (conn->acc_auth)
+	{
+		char * pvalue = conn->acc_auth;
+		char * comma = pvalue;
+		while ((comma = strchr(pvalue, ',')))
+		{
+			*comma = '\0';
+			if (strcmp(pvalue, "password") != 0
+&& strcmp(pvalue, "md5") != 0
+&& strcmp(pvalue, "scram") != 0
+&& strcmp(pvalue, "gss") != 0
+&& strcmp(pvalue, "sspi") != 0)
+			{
+conn->status = CONNECTION_BAD;
+printfPQExpBuffer(>errorMessage,
+			libpq_gettext("invalid authtype value: \"%s\"\n"),
+			  pvalue);
+return false;
+			}
+			*comma = ',';
+			pvalue = comma + 1;
+		}
+		if (strcmp(pvalue, "password") != 0
+			&& strcmp(pvalue, "md5") != 0
+			&& strcmp(pvalue, "scram") != 0
+			&& strcmp(pvalue, "gss") != 0
+			&& strcmp(pvalue, "sspi") != 0)
+		{
+			conn->status = CONNECTION_BAD;
+			p

[HACKERS] A mistake in a comment

2017-06-26 Thread Victor Drobny 

Hello,

I believe that I have found a mistake in a comment to 
parse_phrase_operator function. The comment has the following line:

a  b (distance is no greater than X)
which is not. According to documentation and practical results, this 
line should me changed on something like:

a  b (distance is equal to X)

Patch in the attachments fixes the issue.

Thank you for attention!

Best,
Victordiff --git a/src/backend/utils/adt/tsquery.c b/src/backend/utils/adt/tsquery.c
index ee047bd..260d780 100644
--- a/src/backend/utils/adt/tsquery.c
+++ b/src/backend/utils/adt/tsquery.c
@@ -113,7 +113,7 @@ get_modifiers(char *buf, int16 *weight, bool *prefix)
  * Parse phrase operator. The operator
  * may take the following forms:
  *
- *		a  b (distance is no greater than X)
+ *		a  b (distance is equal to X)
  *		a <-> b (default distance = 1)
  *
  * The buffer should begin with '<' char

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers