Re: [HACKERS] list of credits for release notes

[Dang Minh Huong]

2017/09/28 3:47、Peter Eisentraut <peter.eisentr...@2ndquadrant.com> のメッセージ:

> At the PGCon Developer Meeting it was agreed[0] to add a list of credits
> to the release notes, including everyone who was mentioned in a commit
> message.  I have now completed that list.
> 
> Attached is the proposed documentation commit as well as the raw list.

Thanks! 
Sorry but please change "Huong Dangminh"(my name) to "Dang Minh Huong".

---
Dang Minh Huong

Re: [HACKERS] Extra Vietnamese unaccent rules

[Dang Minh Huong]

Thanks!


On 2017/08/17 11:56, Tom Lane wrote:

Michael Paquier  writes:

On Thu, Aug 17, 2017 at 6:01 AM, Tom Lane  wrote:

I'm not really qualified to review the Python coding
style, but I did fix a typo in a comment.

No pythonist here, but a large confusing "if" condition without any
comments is better if split up and explained with comments if that can
help in clarifying what the code is doing in any language, so thanks
for keeping the code intact.

Certainly agreed on splitting up the logic into multiple statements.
I just meant that I don't know enough Python to know if there are
better ways to do these tests.  (It probably doesn't matter, since
performance of this script is not an issue, and it's not likely to
undergo a lot of further development either.)

regards, tom lane




--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Extra Vietnamese unaccent rules

[Dang Minh Huong]

On 2017/07/05 15:28, Michael Paquier wrote:

I have finally been able to look at this patch.


Thanks for reviewing and the new version of the patch.

(Surprised to see that generate_unaccent_rules.py is inconsistent on
MacOS, runs fine on Linux).

  def get_plain_letter(codepoint, table):
  """Return the base codepoint without marks."""
  if is_letter_with_marks(codepoint, table):
-return table[codepoint.combining_ids[0]]
+if len(table[codepoint.combining_ids[0]].combining_ids) > 1:
+# Recursive to find the plain letter
+return get_plain_letter(table[codepoint.combining_ids[0]],table)
+elif is_plain_letter(table[codepoint.combining_ids[0]]):
+return table[codepoint.combining_ids[0]]
+else:
+return None
  elif is_plain_letter(codepoint):
  return codepoint
  else:
-raise "mu"
+return None
The code paths returning None should not be reached, so I would
suggest adding an assertion instead. Callers of get_plain_letter would
blow up on None, still that would make future debugging harder.

  def is_letter_with_marks(codepoint, table):
-"""Returns true for plain letters combined with one or more marks."""
+"""Returns true for letters combined with one or more marks."""
  # See 
http://www.unicode.org/reports/tr44/tr44-14.html#General_Category_Values
  return len(codepoint.combining_ids) > 1 and \
-   is_plain_letter(table[codepoint.combining_ids[0]]) and \
+   (is_plain_letter(table[codepoint.combining_ids[0]]) or\
+is_letter_with_marks(table[codepoint.combining_ids[0]],table))
and \
 all(is_mark(table[i]) for i in codepoint.combining_ids[1:]
This was already hard to follow, and this patch makes its harder. I
think that the thing should be refactored with multiple conditions.

  if is_letter_with_marks(codepoint, table):
-charactersSet.add((codepoint.id,
+if get_plain_letter(codepoint, table) <> None:
+charactersSet.add((codepoint.id,
This change is not necessary as a letter with marks is not a plain
character anyway.

Testing with characters having two accents, the results are produced
as wanted. I am attaching an updated patch with all those
simplifications. Thoughts?


Thanks, so pretty. The patch is fine to me.

---
Thanks and best regards,
Dang Minh Huong

---
このEメールはアバスト アンチウイルスによりウイルススキャンされています。
https://www.avast.com/antivirus



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Extra Vietnamese unaccent rules

[Dang Minh Huong]

On Jun 4, 29 Heisei, at 00:48, Bruce Momjian <br...@momjian.us> wrote:On Sun, Jun  4, 2017 at 12:43:17AM +0900, Dang Minh Huong wrote:On May 30, 29 Heisei, at 00:22, Dang Minh Huong <kakalo...@gmail.com> wrote:On May 29, 29 Heisei, at 10:47, Thomas Munro <thomas.mu...@enterprisedb.com <mailto:thomas.mu...@enterprisedb.com>> wrote:On Sun, May 28, 2017 at 7:55 PM, Dang Minh Huong <kakalo...@gmail.com <mailto:kakalo...@gmail.com>> wrote:Thanks for reporting and lecture about unicode.I attached a patch as the instruction from Thomas. Could you confirm it.-   is_plain_letter(table[codepoint.combining_ids[0]]) and \+   (is_plain_letter(table[codepoint.combining_ids[0]]) or\+    len(table[codepoint.combining_ids[0]].combining_ids) > 1) and \Shouldn't you use "or is_letter_with_marks()", instead of "or len(...)1"?  Your test might catch something that isn't based on a 'letter'(according to is_plain_letter).  Otherwise this looks pretty good tome.  Please add it to the next commitfest.Thanks for confirm, sir.I will add it to the next CF soon.Sorry for lately response. I attach the update patch.Uh, there is no patch attached.Sorry sir, reattach the patch.I also added it to the next CF and set reviewers to Thomas Munro. Could you confirm for me.

unaccent.patch
Description: Binary data
---Thanks and best regards,Dang Minh Huong

Re: [HACKERS] Extra Vietnamese unaccent rules

[Dang Minh Huong]

On May 30, 29 Heisei, at 00:22, Dang Minh Huong <kakalo...@gmail.com> wrote:

unaccent.patch
Description: Binary data
On May 29, 29 Heisei, at 10:47, Thomas Munro <thomas.mu...@enterprisedb.com> wrote:On Sun, May 28, 2017 at 7:55 PM, Dang Minh Huong <kakalo...@gmail.com> wrote:Thanks for reporting and lecture about unicode.I attached a patch as the instruction from Thomas. Could you confirm it.-   is_plain_letter(table[codepoint.combining_ids[0]]) and \+   (is_plain_letter(table[codepoint.combining_ids[0]]) or\+    len(table[codepoint.combining_ids[0]].combining_ids) > 1) and \Shouldn't you use "or is_letter_with_marks()", instead of "or len(...)1"?  Your test might catch something that isn't based on a 'letter'(according to is_plain_letter).  Otherwise this looks pretty good tome.  Please add it to the next commitfest.Thanks for confirm, sir.I will add it to the next CF soon.Sorry for lately response. I attach the update patch.---Thanks and best regards,Dang Minh Huong

Re: [HACKERS] Extra Vietnamese unaccent rules

[Dang Minh Huong]

> On May 29, 29 Heisei, at 10:47, Thomas Munro <thomas.mu...@enterprisedb.com> 
> wrote:
> 
> On Sun, May 28, 2017 at 7:55 PM, Dang Minh Huong <kakalo...@gmail.com> wrote:
>> Thanks for reporting and lecture about unicode.
>> I attached a patch as the instruction from Thomas. Could you confirm it.
> 
> -   is_plain_letter(table[codepoint.combining_ids[0]]) and \
> +   (is_plain_letter(table[codepoint.combining_ids[0]]) or\
> +len(table[codepoint.combining_ids[0]].combining_ids) > 1) and \
> 
> Shouldn't you use "or is_letter_with_marks()", instead of "or len(...)
>> 1"?  Your test might catch something that isn't based on a 'letter'
> (according to is_plain_letter).  Otherwise this looks pretty good to
> me.  Please add it to the next commitfest.

Thanks for confirm, sir.
I will add it to the next CF soon.

> I expect that some users in Vietnam will consider this to be a bugfix,
> which raises the question of whether to backpatch it.  Perhaps we
> could consider fixing it for 10.  Then users of older versions could
> grab the rules file from 10 to use with 9.whatever if they want to do
> that and reindex their data as appropriate.

I am also inclined to the fixing it for 10, because it will not affect to 
current users.
But do you want to back-patch to all supported versions Kha Nguyen?
# I would also want to note that, not only Vietnamese characters were missed to 
add from the rule list.


---
Thanks and best regards,
Dang Minh Huong

Re: [HACKERS] Extra Vietnamese unaccent rules

[Dang Minh Huong]

Hi,

unaccent.patch
Description: Binary data
I am interested in this thread.On May 27, 29 Heisei, at 10:41, Michael Paquier <michael.paqu...@gmail.com> wrote:On Fri, May 26, 2017 at 5:48 PM, Thomas Munro<thomas.mu...@enterprisedb.com> wrote:Unicode has two ways to represent characters with accents: either withcomposed codepoints like "é" or decomposed codepoints where you say"e" and then "´".  The field "00E2 0301" is the decomposed form ofthat character above.  Our job here is to identify the basic letterthat each composed character contains, by analysing the decomposedfield that you see in that line.  I failed to realise that characterswith TWO accents are described as a composed character with ONE accentplus another accent.Doesn't that depend on the NF operation you are working on? With acanonical decomposition it seems to me that a character with twoaccents can as well be decomposed with one character and two composingcharacter accents (NFKC does a canonical decomposition in one of itssteps).You don't have to worry about decoding that line, it's all done inthat Python script.  The problem is just in the functionis_letter_with_marks().  Instead of just checking if combining_ids[0]is a plain letter, it looks like it should also check ifcombining_ids[0] itself is a letter with marks.  Also get_plain_letterwould need to be able to recurse to extract the "a".Thanks for reporting and lecture about unicode.I attached a patch as the instruction from Thomas. Could you confirm it.Actually, with the recent work that has been done withunicode_norm_table.h which has been to transpose UnicodeData.txt intouser-friendly tables, shouldn't the python script of unaccent/ bereplaced by something that works on this table? This does a canonicaldecomposition but just keeps the first characters with a classordering of 0. So we have basic APIs able to look at UnicodeData.txtand let caller do decision making with the result returned.-- MichaelThanks, i will learning about it.---Dang Minh Huong

[HACKERS] Duplicated row after promote in synchronous streaming replication

[Dang Minh Huong]

Hi all,

I'm using PostgreSQL 9.1.10 for my HA project and have found this problem.

I did (multiple times) the following sequence in my primary/standby synchronous 
replication environment,

1. Update rows in a table (which have primary key constraint column) in active 
DB

2. Stop active DB

3. Promote standby DB

4. Confirm the updated table in promoted standby (new primary) and found that, 
there's a duplicate updated row (number of row was increased).

I think it is a replication bug but wonder if it was fixed yet. 
Can somebody help me?

I'm not yet confirm PostgreSQL source, but here is my investigation result.

Updated table before promoted were HOT update (index file was not changed).

After promote i continue update that duplicated row (it returned two row 
updated), and confirm with pg_filedump, i found the duplicated row and only one 
is related to primary key index constraint.

Compare with old active DB, i saw that after promote line pointer of updated 
row (duplicated row) is broken into two line pointer, the new one is related to 
primary index constraint and the other is not related to. Some thing like 
below, 

Old active DB:
ctid(0,3)-ctid(0,6)-ctid(0,7)

New active DB (after promote and update):
ctid(0,3)-ctid(0,9)
ctid(0,7)-ctid(0,10)

ctid(0,10) is not related to primary key index constraint.

Is something was wrong in redo log in standby DB? Or line pointer in HOT update 
feature?

Thanks and best regards,
Huong,

Re: [HACKERS] Duplicated row after promote in synchronous streaming replication

[Dang Minh Huong]

2014/03/27 0:18、Thom Brown t...@linux.com のメッセージ:

 On 26 March 2014 15:08, Dang Minh Huong kakalo...@gmail.com wrote:
 Hi all,
 
 I'm using PostgreSQL 9.1.10 for my HA project and have found this problem.
 
 I did (multiple times) the following sequence in my primary/standby
 synchronous replication environment,
 
 1. Update rows in a table (which have primary key constraint column) in
 active DB
 
 2. Stop active DB
 
 3. Promote standby DB
 
 4. Confirm the updated table in promoted standby (new primary) and found
 that, there's a duplicate updated row (number of row was increased).
 
 I think it is a replication bug but wonder if it was fixed yet.
 Can somebody help me?
 
 I'm not yet confirm PostgreSQL source, but here is my investigation result.
 
 Updated table before promoted were HOT update (index file was not changed).
 
 After promote i continue update that duplicated row (it returned two row
 updated), and confirm with pg_filedump, i found the duplicated row and only
 one is related to primary key index constraint.
 
 Compare with old active DB, i saw that after promote line pointer of updated
 row (duplicated row) is broken into two line pointer, the new one is related
 to primary index constraint and the other is not related to. Some thing like
 below,
 
 Old active DB:
 ctid(0,3)-ctid(0,6)-ctid(0,7)
 
 New active DB (after promote and update):
 ctid(0,3)-ctid(0,9)
 ctid(0,7)-ctid(0,10)
 
 ctid(0,10) is not related to primary key index constraint.
 
 Is something was wrong in redo log in standby DB? Or line pointer in HOT
 update feature?
 
 It sounds like you're hitting a bug that was introduced in that
 exact minor version, and has since been fixed:
 
 http://www.postgresql.org/docs/9.1/static/release-9-1-11.html
 

Thanks for your prompt response. I will confirm and revision-up if it is needed.

 You should update to the latest minor version, then re-base your
 standbys from the primary.
 
 -- 
 Thom


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [BUGS] replication_timeout not effective

[Dang Minh Huong]

Hi Amit,

Thank you for your consideration.

My project not allows to use 9.2 or 9.3.

In 9.3, it sounds replication_timeout is replaced by wal_sender_timeout. 
So if it is solved in 9.3 i think there is a way to terminate it. 
I hope it is fixed in 9.1 soon

Regards,

2013/04/10 18:33、Amit Kapila amit.kap...@huawei.com のメッセージ:

 Sent: Wednesday, April 10, 2013 1:49 PM Dang Minh Huong wrote:
 To: Amit Kapila
 Subject: Re: [BUGS] replication_timeout not effective
 On Wednesday, April 10, 2013 1:49 PM
 Hi,
 
 Thank you for your soon reply.
 
 I'm trying to set the network timeout related parameters to terminate
 it.
 
 # i've tried to set postgresql.conf's tcp_keepalives_* but not success.
 
 I have also tried those, but they didn't work that's why I have proposed
 this feature in 9.3
 
 Please send mail on community list, others can also help you if they have
 any idea for avoiding such problems.
 
 2013/04/10 14:05、Amit Kapila amit.kap...@huawei.com のメッセージ:
 
 On Wednesday, April 10, 2013 9:35 AM Dang Minh Huong wrote:
 Hi,
 
 I'm wondering  if this is a bug of PostgreSQL.
 
 PostgreSQL's show that replication_timeout parameter can Terminate
 replication connections that are inactive longer than the specified
 number of milliseconds. But in my environment the sender process  is
 hang up (in several tens of minunites) if i turn off  (by power off)
 Standby PC while pg_basebackup is excuting.
 
 Is this correct?
 
 As my debug, sender process is terminated when recieve SIGPIPE
 process but it come too slow (about 30minutes after standby PC was
 down).
 
 For such scenario's, new parameter wal_sender_timeout has been
 introduced in 9.3. Refer below:
 http://www.postgresql.org/docs/devel/static/runtime-config-
 replication.html#RUNTIME-CONFIG-REPLICATION-SENDER
 
 I am not sure how to get rid of this problem in 9.1.9
 
 With Regards,
 Amit Kapila.
 


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [BUGS] replication_timeout not effective

[Dang Minh Huong]

Thanks all,

(2013/04/10 22:55), Andres Freund wrote:

On 2013-04-10 22:38:07 +0900, Kyotaro HORIGUCHI wrote:

Hello,

On Wed, Apr 10, 2013 at 6:57 PM, Dang Minh Huong kakalo...@gmail.com wrote:

In 9.3, it sounds replication_timeout is replaced by wal_sender_timeout.
So if it is solved in 9.3 i think there is a way to terminate it.
I hope it is fixed in 9.1 soon

Hmm. He said that,


But in my environment the sender process is hang up (in several tens of 
minunites) if i turn off  (by power off) Standby PC while *pg_basebackup* is 
excuting.

Does basebackup run only on 'replication connection' ?
As far as I saw base backup uses 'base backup' connection in addition
to 'streaming' connection. The former seems not under the control of
wal_sender_timeout or replication_timeout and easily blocked at
send(2) after sudden cut out of the network connection underneath.
Although the latter indeed is terminated by them.

Yes, it's run via a walsender connection. The only problem is that it
doesn't check for those timeouts. I am not sure it would be a good thing
to do so to be honest. At least not using the same timeout as actual WAL
sending, thats just has different characteristics.
On the other hand, hanging around that long isn't nice either...

I tried max_wal_sender with 1, so when the walsender is hanging.
I can not run again pg_basebackup (or start the standby DB).
I'm increasing it to 2, so the seconds successfully. But i'm afraid
 that when the third occures the hanging walsender in the first
 is not yet terminated...

 I think not, but is there a way to terminate hanging up but not
 restart PostgreSQL server or kill walsender process?
 (kill walsender process can caused a crash to DB server,
 so i don't want to do it).

 # i've also tried with pg_cancel_backend() but it did not work too.

Blocking in send(2) might could occur for async-rep connection but not
likely for sync-rep since it does not fill the buffers of libpq and
socket easilly.

You just need larger transactions for it. A COPY or so ought to do it.

Greetings,

Andres Freund


Regard,
Huong DM


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [BUGS] replication_timeout not effective

[Dang Minh Huong]

2013/04/10 23:44、Andres Freund and...@2ndquadrant.com のメッセージ:

 On 2013-04-10 23:37:44 +0900, Dang Minh Huong wrote:
 Thanks all,
 
 (2013/04/10 22:55), Andres Freund wrote:
 On 2013-04-10 22:38:07 +0900, Kyotaro HORIGUCHI wrote:
 Hello,
 
 On Wed, Apr 10, 2013 at 6:57 PM, Dang Minh Huong kakalo...@gmail.com 
 wrote:
 In 9.3, it sounds replication_timeout is replaced by wal_sender_timeout.
 So if it is solved in 9.3 i think there is a way to terminate it.
 I hope it is fixed in 9.1 soon
 Hmm. He said that,
 
 But in my environment the sender process is hang up (in several tens of 
 minunites) if i turn off  (by power off) Standby PC while *pg_basebackup* 
 is excuting.
 Does basebackup run only on 'replication connection' ?
 As far as I saw base backup uses 'base backup' connection in addition
 to 'streaming' connection. The former seems not under the control of
 wal_sender_timeout or replication_timeout and easily blocked at
 send(2) after sudden cut out of the network connection underneath.
 Although the latter indeed is terminated by them.
 Yes, it's run via a walsender connection. The only problem is that it
 doesn't check for those timeouts. I am not sure it would be a good thing
 to do so to be honest. At least not using the same timeout as actual WAL
 sending, thats just has different characteristics.
 On the other hand, hanging around that long isn't nice either...
 I tried max_wal_sender with 1, so when the walsender is hanging.
 I can not run again pg_basebackup (or start the standby DB).
 I'm increasing it to 2, so the seconds successfully. But i'm afraid
 that when the third occures the hanging walsender in the first
 is not yet terminated...
 
 I think not, but is there a way to terminate hanging up but not
 restart PostgreSQL server or kill walsender process?
 (kill walsender process can caused a crash to DB server,
 so i don't want to do it).
 
 Depending on where its hanging a normal SELECT
 pg_terminate_backend(pid); might do it.
 
Greate! it worked. Thank you very much.

 Otherwise you will have to wait for the operating system's tcp timeout.
 
 Greetings,
 
 Andres Freund
 
 -- 
 Andres Freund   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training  Services

Regards,
Huong DM

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers