Re: [pgAdmin4][RM#3289] Can't query SQL_ASCII database.

[Aditya Toshniwal]

Hi

On Tue, Jun 5, 2018 at 1:08 AM, Joao De Almeida Pereira <
jdealmeidapere...@pivotal.io> wrote:

> Hello Aditya,
>
>>
>>
>> There is no change related to notifications in this patch.
>> The below code is minor fix related to connection status of sql editor.
>> Can you please share the code snippet if it is not the below.
>>
>> -# Check for the asynchronous notifies statements.
>> -conn.check_notifies(True)
>> -notifies = conn.get_notifies()
>> +if status is not None:
>> +# Check for the asynchronous notifies statements.
>> +conn.check_notifies(True)
>> +notifies = conn.get_notifies()
>>
>>
> This is a minor fix, but is it related to querying SQL_ASCII database?
>
No its not. It is something I found when I was working on SQL_ASCII related
changes.
Well then, will send a separate patch for it.

>
> Thanks
> Victoria && Joao
>

[pgAdmin4][patch] sqleditor connection status minor patch

[Aditya Toshniwal]

Hi Hackers,

PFA a minor patch for sqleditor. When you disconnect the server with an
open sqleditor tab, exception occurs at the back end. Also, after
connecting server, the sqleditor is not able to connect back because of the
exception. The patch is to handle it.
Kindly review.

Thanks and Regards,
Aditya Toshniwal
Software Engineer | EnterpriseDB Software Solutions | Pune
"Don't Complain about Heat, Plant a tree"
diff --git a/web/pgadmin/tools/sqleditor/__init__.py b/web/pgadmin/tools/sqleditor/__init__.py
index 013e4dc..a9460dd 100644
--- a/web/pgadmin/tools/sqleditor/__init__.py
+++ b/web/pgadmin/tools/sqleditor/__init__.py
@@ -1480,19 +1480,24 @@ def query_tool_status(trans_id):
 if conn and trans_obj and session_obj:
 status = conn.transaction_status()
 
-# Check for the asynchronous notifies statements.
-conn.check_notifies(True)
-notifies = conn.get_notifies()
+if status is not None:
+# Check for the asynchronous notifies statements.
+conn.check_notifies(True)
+notifies = conn.get_notifies()
 
-return make_json_response(
-data={
-'status': status,
-'message': gettext(
-CONNECTION_STATUS_MESSAGE_MAPPING.get(status),
-),
-'notifies': notifies
-}
-)
+return make_json_response(
+data={
+'status': status,
+'message': gettext(
+CONNECTION_STATUS_MESSAGE_MAPPING.get(status),
+),
+'notifies': notifies
+}
+)
+else:
+return internal_server_error(
+errormsg=gettext("Transaction status check failed.")
+)
 else:
 return internal_server_error(
 errormsg=gettext("Transaction status check failed.")

Re: pgAgent 4.0 patch

[Ashesh Vashi]

On Mon, Jun 4, 2018 at 10:04 PM, Dave Page  wrote:

> Hi
>
> Can you review the attached please Ashesh?
>
Hi Dave,

Due to the wrong pattern in '.gitignore', it was ignoring the new file
'sql/pgagent--3.4--4.0.sql'.
We should ignore only the '*.sql' in the root directory.
Something like this.

*$ git diff .gitignore*
*diff --git a/.gitignore b/.gitignore*
*index 9883354..40fe7a9 100644*
*--- a/.gitignore*
*+++ b/.gitignore*
*@@ -6,7 +6,12 @@ CPackSourceConfig.cmake*
* Makefile*
* cmake_install.cmake*
* install_manifest.txt*
*-pgagent*
*+# Ignore 'pgagent' & 'pgagent.control' in root directory only.*
*+/pgagent*
*+!/*/pgagent*
* pgagent.control*
*+!/*/pgagent.control*
* *.log*
*-*.sql*
*+# Ignore '*.sql' files in root directory only.*
*+/*.sql*
*+!/*/*.sql*


And, I am getting the following warning applying the patch.

*$ git apply ~/Downloads/pgagent-4.0-update.diff*
*/Users/asheshvashi/Downloads/pgagent-4.0-update.diff:320: trailing
whitespace.*
*//*
*warning: 1 line adds whitespace errors.*


Otherwise - it looks good to me.

--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company



*http://www.linkedin.com/in/asheshvashi
*

>
> Thanks.
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

Re: [pgadmin4][Patch]: Test cases for the backup module

[Khushboo Vashi]

On Mon, Jun 4, 2018 at 8:41 PM, Joao De Almeida Pereira <
jdealmeidapere...@pivotal.io> wrote:

> Hi Khushboo,
>
> Some tests are failing in greenplum:
> https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/
> pipelines/pgadmin-patch/jobs/run-tests/builds/108
> The piece of code responsible for the error is:
>
> if server['default_binary_paths'] is not None:
> test_utils.set_preference(server['default_binary_paths'])
>
> config.DEFAULT_BINARY_PATHS = {
> "pg": str(server['default_binary_paths']['pg']),
> "ppas": str(server['default_binary_paths']['ppas']),
> "gpdb": ""
> }
>
>
> Can you send me the test_config.json file?  The above code sets the paths
to the SQLite database and through the logs couldn't figure out the exact
failure.

> test_backup_utils.py file name is misleading, these are not tests, are
> helpers.
> ​
>
>
> Thanks
> Victoria & Joao
>
> On Mon, Jun 4, 2018 at 1:36 AM Khushboo Vashi <
> khushboo.va...@enterprisedb.com> wrote:
>
>>
>>
>> On Sat, Jun 2, 2018 at 3:01 AM, Dave Page  wrote:
>>
>>> Hi
>>>
>>> This looks good, except that it's leaving the test_restore_database
>>> behind. Can we clean that up please?
>>>
>>> PFA updated patch.
>>
>>> Thanks.
>>>
>>> On Fri, Jun 1, 2018 at 7:06 AM, Khushboo Vashi <
>>> khushboo.va...@enterprisedb.com> wrote:
>>>
 Hi Victoria,

 Thanks for reviewing the patch.
 The tests were failing due to the latest commit #
 2b4605a9d390cb44e5dfe9967c3adf2b28d04f1f  - Ensure
 backup/restore/maintenance work via SSH tunnels. Fixes #3355

 I have fixed the issues and attached the updated patch.

 Thanks,
 Khushboo

 On Thu, May 31, 2018 at 10:00 PM, Victoria Henry 
 wrote:

> Hi there,
>
> We've been noticing some issues with the tests on both our CI and
> local Mac workstations.
>
>1. When the following code blocks are invoked - we get plenty of
>app.context() issues. It must not be valid when running tests.
>
> ​
>
> from pgadmin.utils.driver import get_driver
> driver = get_driver(PG_DEFAULT_DRIVER)
> manager = driver.connection_manager(self.sid)
>
> host = manager.local_bind_host if manager.use_ssh_tunnel else s.host
> port = manager.local_bind_port if manager.use_ssh_tunnel else s.port
>
> 2. When we finally enable
>
> "default_binary_paths": {
>
> in our test_config, we get more failing tests that look like:
>
> ==
> FAIL: runTest 
> (pgadmin.tools.restore.tests.test_restore_create_job_unit_test.RestoreCreateJobTest)
> When restore object with option - Miscellaneous
> --
> Traceback (most recent call last):
>   File 
> "/Users/pivotal/.pyenv/versions/3.6.5/lib/python3.6/unittest/mock.py", 
> line 1179, in patched
> return func(*args, **keywargs)
>   File 
> "/Users/pivotal/workspace/pgadmin4/web/pgadmin/tools/restore/tests/test_restore_create_job_unit_test.py",
>  line 295, in runTest
> self.assertEquals(response.status_code, 200)
> AssertionError: 410 != 200
>
> And
>
> When restore object with the sections options ... 2018-05-31 
> 12:24:42,988: ERRORpgadmin:illegal environment variable name
> Traceback (most recent call last):
>   File 
> "/Users/pivotal/workspace/pgadmin4/web/pgadmin/tools/restore/__init__.py",
>  line 352, in create_restore_job
> manager.export_password_env(p.id)
>   File 
> "/Users/pivotal/workspace/pgadmin4/web/pgadmin/utils/driver/psycopg2/server_manager.py",
>  line 365, in export_password_env
> os.environ[str(env)] = password
>   File "/Users/pivotal/.pyenv/versions/3.6.5/lib/python3.6/os.py", line 
> 675, in __setitem__
> self.putenv(key, value)
> ValueError: illegal environment variable name
> FAIL
>
> ​
>
> Sincerely,
>
> Victoria && Anthony
>
> On Thu, May 31, 2018 at 1:16 AM Khushboo Vashi <
> khushboo.va...@enterprisedb.com> wrote:
>
>> Hi,
>>
>> Please find the attached updated patch with the fixes.
>> The test cases were only failing on MAC not on Linux.
>>
>> Thanks,
>> Khushboo
>>
>> On Wed, May 30, 2018 at 10:13 AM, Khushboo Vashi <
>> khushboo.va...@enterprisedb.com> wrote:
>>
>>>
>>>
>>> On Wed, May 30, 2018 at 1:05 AM, Dave Page 
>>> wrote:
>>>
 Hi

 On Mon, May 28, 2018 at 8:09 AM, Khushboo Vashi <
 khushboo.va...@enterprisedb.com> wrote:

> Hi,
>
> please find the attached updated patch for the test cases of
> Backup, Restore and Maintenance modules which includes:
>
> 1. Unit test cases
> 2. End to end regression test cases
> 

Re: [pgAdmin4][RM#3289] Can't query SQL_ASCII database.

[Joao De Almeida Pereira]

Hello Aditya,

>
>
> There is no change related to notifications in this patch.
> The below code is minor fix related to connection status of sql editor.
> Can you please share the code snippet if it is not the below.
>
> -# Check for the asynchronous notifies statements.
> -conn.check_notifies(True)
> -notifies = conn.get_notifies()
> +if status is not None:
> +# Check for the asynchronous notifies statements.
> +conn.check_notifies(True)
> +notifies = conn.get_notifies()
>
>
This is a minor fix, but is it related to querying SQL_ASCII database?

Thanks
Victoria && Joao

Re: [pgAdmin4][RM#3289] Can't query SQL_ASCII database.

[Aditya Toshniwal]

Hi Victoria/Joao,

There is no change related to notifications in this patch.
The below code is minor fix related to connection status of sql editor. Can
you please share the code snippet if it is not the below.

-# Check for the asynchronous notifies statements.
-conn.check_notifies(True)
-notifies = conn.get_notifies()
+if status is not None:
+# Check for the asynchronous notifies statements.
+conn.check_notifies(True)
+notifies = conn.get_notifies()

The test cases work perfectly fine on my machine. I will try if I can
simulate the failure on some other machines.


Thanks and Regards,
Aditya Toshniwal
Software Engineer | EnterpriseDB Software Solutions | Pune
"Don't Complain about Heat, Plant a tree"

On Mon, Jun 4, 2018 at 8:54 PM, Joao De Almeida Pereira <
jdealmeidapere...@pivotal.io> wrote:

> Hi Aditya,
> Looks like there are changes in this patch that are related to
> notifications, these should go into a separate commit as they are not
> related to encoding.
> Also the tests are failing in our pipeline: https://gpdb-dev.
> bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-
> patch/jobs/run-tests/builds/110
>
> Thanks
> Victoria && Joao
>
> On Mon, Jun 4, 2018 at 5:55 AM Aditya Toshniwal  enterprisedb.com> wrote:
>
>> Hi Hackers,
>>
>> Please ignore previous patch. Fixed some linter issues. PFA updated patch.
>>
>> Thanks and Regards,
>> Aditya Toshniwal
>> Software Engineer | EnterpriseDB Software Solutions | Pune
>> "Don't Complain about Heat, Plant a tree"
>>
>> On Mon, Jun 4, 2018 at 10:53 AM, Aditya Toshniwal > enterprisedb.com> wrote:
>>
>>> Hi Hackers,
>>>
>>> PFA the updated patch on this. I have tried to add test cases to check
>>> for different encoding database. In the test run, it will create a
>>> database, fire a query for a string, check if we get the output and drops
>>> the database.
>>> Kindly review.
>>>
>>> Thanks and Regards,
>>> Aditya Toshniwal
>>> Software Engineer | EnterpriseDB Software Solutions | Pune
>>> "Don't Complain about Heat, Plant a tree"
>>>
>>> On Thu, May 31, 2018 at 6:42 PM, Dave Page  wrote:
>>>
 Hi

 On Thu, May 31, 2018 at 1:20 AM, Aditya Toshniwal >>> enterprisedb.com> wrote:

> Hi Victoria/Joao,
>
> On Thu, May 31, 2018 at 2:06 AM, Joao De Almeida Pereira <
> jdealmeidapere...@pivotal.io> wrote:
>
>> Hello Aditya,
>>
>> It looks ok and it passes CI.
>>
>> We have some recommendations:
>> - These look like 2 different changes so they should be in separated
>> commits
>>
>
> If you are talking of set client_encoding, then its not a bug. Its a
> choice given to Postgres DB user to change the encoding of the characters.
> Postgres will translate characters from Server Encoding to Client 
> Encoding,
> and will throw error like mentioned previously. This link will help better
> - https://www.postgresql.org/docs/10/static/multibyte.html
> The actual bug was, even after changing the client encoding to
> SQL_ASCII, pgAdmin4 was not able to show the output as it was failing in
> encoding by psycopg2. The patch is for resolving that.
>
>
>> - Do we have test coverage for the bug that you are talking about? If
>> not we should, to ensure this problem does not happen again in a future
>> change.
>>
>
> It is not possible adding test cases for encoding related stuff, as
> Postgres support a lot many different types of encoding and conversions
> (refer above link)
>

 I was going to ask the same thing. Per https://www.postgresql.
 org/docs/10/static/multibyte.html#id-1.6.10.5.7, every characterset
 except SQL_ASCII can be converted to UTF-8, so we only need to test that
 UTF-8 and some other charactersets besides SQL_ASCII work, and then
 separately that SQL_ASCII with characters known not to be in UTF-8 work.


>
>> Thanks
>> Victoria && Joao
>>
>> On Wed, May 30, 2018 at 3:06 AM Aditya Toshniwal > enterprisedb.com> wrote:
>>
>>> Hi Hackers,
>>>
>>> PFA updated patch after all the permutations, combinations for
>>> encoding for SQL_ASCII database.  Also fixed a small glitch for sql
>>> editor connection status check.
>>>
>>> Please note, ERROR: invalid byte sequence for encoding "UTF8": 0xe9
>>> 0x73 is a Postgres DB error and not pgAdmin4 error.
>>>
>>> 
>>>
>>> You need to change client_encoding to the appropriate. After
>>> changing client_encoding using command - set client_encoding='XYZ', it 
>>> will
>>> give not give error.
>>>
>>> 
>>>
>>> Thanks and Regards,
>>> Aditya Toshniwal
>>> Software Engineer | EnterpriseDB Software Solutions | Pune
>>> "Don't Complain about Heat, Plant a tree"
>>>
>>> On Wed, May 23, 2018 at 10:13 AM, Aditya Toshniwal <
>>> 

pgAgent 4.0 patch

[Dave Page]

Hi

Can you review the attached please Ashesh?

Thanks.

-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


pgagent-4.0-update.diff
Description: Binary data

Re: [pgAdmin4][RM#3289] Can't query SQL_ASCII database.

[Joao De Almeida Pereira]

Hi Aditya,
Looks like there are changes in this patch that are related to
notifications, these should go into a separate commit as they are not
related to encoding.
Also the tests are failing in our pipeline:
https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-tests/builds/110

Thanks
Victoria && Joao

On Mon, Jun 4, 2018 at 5:55 AM Aditya Toshniwal <
aditya.toshni...@enterprisedb.com> wrote:

> Hi Hackers,
>
> Please ignore previous patch. Fixed some linter issues. PFA updated patch.
>
> Thanks and Regards,
> Aditya Toshniwal
> Software Engineer | EnterpriseDB Software Solutions | Pune
> "Don't Complain about Heat, Plant a tree"
>
> On Mon, Jun 4, 2018 at 10:53 AM, Aditya Toshniwal <
> aditya.toshni...@enterprisedb.com> wrote:
>
>> Hi Hackers,
>>
>> PFA the updated patch on this. I have tried to add test cases to check
>> for different encoding database. In the test run, it will create a
>> database, fire a query for a string, check if we get the output and drops
>> the database.
>> Kindly review.
>>
>> Thanks and Regards,
>> Aditya Toshniwal
>> Software Engineer | EnterpriseDB Software Solutions | Pune
>> "Don't Complain about Heat, Plant a tree"
>>
>> On Thu, May 31, 2018 at 6:42 PM, Dave Page  wrote:
>>
>>> Hi
>>>
>>> On Thu, May 31, 2018 at 1:20 AM, Aditya Toshniwal <
>>> aditya.toshni...@enterprisedb.com> wrote:
>>>
 Hi Victoria/Joao,

 On Thu, May 31, 2018 at 2:06 AM, Joao De Almeida Pereira <
 jdealmeidapere...@pivotal.io> wrote:

> Hello Aditya,
>
> It looks ok and it passes CI.
>
> We have some recommendations:
> - These look like 2 different changes so they should be in separated
> commits
>

 If you are talking of set client_encoding, then its not a bug. Its a
 choice given to Postgres DB user to change the encoding of the characters.
 Postgres will translate characters from Server Encoding to Client Encoding,
 and will throw error like mentioned previously. This link will help better
 - https://www.postgresql.org/docs/10/static/multibyte.html
 The actual bug was, even after changing the client encoding to
 SQL_ASCII, pgAdmin4 was not able to show the output as it was failing in
 encoding by psycopg2. The patch is for resolving that.


> - Do we have test coverage for the bug that you are talking about? If
> not we should, to ensure this problem does not happen again in a future
> change.
>

 It is not possible adding test cases for encoding related stuff, as
 Postgres support a lot many different types of encoding and conversions
 (refer above link)

>>>
>>> I was going to ask the same thing. Per
>>> https://www.postgresql.org/docs/10/static/multibyte.html#id-1.6.10.5.7,
>>> every characterset except SQL_ASCII can be converted to UTF-8, so we only
>>> need to test that UTF-8 and some other charactersets besides SQL_ASCII
>>> work, and then separately that SQL_ASCII with characters known not to be in
>>> UTF-8 work.
>>>
>>>

> Thanks
> Victoria && Joao
>
> On Wed, May 30, 2018 at 3:06 AM Aditya Toshniwal <
> aditya.toshni...@enterprisedb.com> wrote:
>
>> Hi Hackers,
>>
>> PFA updated patch after all the permutations, combinations for
>> encoding for SQL_ASCII database.  Also fixed a small glitch for sql
>> editor connection status check.
>>
>> Please note, ERROR: invalid byte sequence for encoding "UTF8": 0xe9
>> 0x73 is a Postgres DB error and not pgAdmin4 error.
>>
>> 
>>
>> You need to change client_encoding to the appropriate. After changing
>> client_encoding using command - set client_encoding='XYZ', it will give 
>> not
>> give error.
>>
>> 
>>
>> Thanks and Regards,
>> Aditya Toshniwal
>> Software Engineer | EnterpriseDB Software Solutions | Pune
>> "Don't Complain about Heat, Plant a tree"
>>
>> On Wed, May 23, 2018 at 10:13 AM, Aditya Toshniwal <
>> aditya.toshni...@enterprisedb.com> wrote:
>>
>>> Thank you Victoria, Anthony.
>>>
>>> Thanks and Regards,
>>> Aditya Toshniwal
>>> Software Engineer | EnterpriseDB Software Solutions | Pune
>>> "Don't Complain about Heat, Plant a tree"
>>>
>>> On Tue, May 22, 2018 at 7:15 PM, Victoria Henry 
>>> wrote:
>>>
 Hi Aditya,

 We made a minor change to make the patch so the python linter can
 pass.  Attached is the change we made.
 Everything else looks good.

 Sincerely,

 Victoria & Anthony

 On Tue, May 22, 2018 at 4:46 AM Aditya Toshniwal <
 aditya.toshni...@enterprisedb.com> wrote:

> Hi,
>
> PFA updated patch. Linter issues are fixed ( we dont have any
> linter setup for python :-( )
> Regarding test cases, they run successfully on my system and the
> reason it 

Re: [pgadmin4][Patch]: Test cases for the backup module

[Joao De Almeida Pereira]

Hi Khushboo,

Some tests are failing in greenplum:
https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-tests/builds/108
The piece of code responsible for the error is:

if server['default_binary_paths'] is not None:
test_utils.set_preference(server['default_binary_paths'])

config.DEFAULT_BINARY_PATHS = {
"pg": str(server['default_binary_paths']['pg']),
"ppas": str(server['default_binary_paths']['ppas']),
"gpdb": ""
}


test_backup_utils.py file name is misleading, these are not tests, are
helpers.
​


Thanks
Victoria & Joao

On Mon, Jun 4, 2018 at 1:36 AM Khushboo Vashi <
khushboo.va...@enterprisedb.com> wrote:

>
>
> On Sat, Jun 2, 2018 at 3:01 AM, Dave Page  wrote:
>
>> Hi
>>
>> This looks good, except that it's leaving the test_restore_database
>> behind. Can we clean that up please?
>>
>> PFA updated patch.
>
>> Thanks.
>>
>> On Fri, Jun 1, 2018 at 7:06 AM, Khushboo Vashi <
>> khushboo.va...@enterprisedb.com> wrote:
>>
>>> Hi Victoria,
>>>
>>> Thanks for reviewing the patch.
>>> The tests were failing due to the latest commit
>>> #2b4605a9d390cb44e5dfe9967c3adf2b28d04f1f  - Ensure
>>> backup/restore/maintenance work via SSH tunnels. Fixes #3355
>>>
>>> I have fixed the issues and attached the updated patch.
>>>
>>> Thanks,
>>> Khushboo
>>>
>>> On Thu, May 31, 2018 at 10:00 PM, Victoria Henry 
>>> wrote:
>>>
 Hi there,

 We've been noticing some issues with the tests on both our CI and local
 Mac workstations.

1. When the following code blocks are invoked - we get plenty of
app.context() issues. It must not be valid when running tests.

 ​

 from pgadmin.utils.driver import get_driver
 driver = get_driver(PG_DEFAULT_DRIVER)
 manager = driver.connection_manager(self.sid)

 host = manager.local_bind_host if manager.use_ssh_tunnel else s.host
 port = manager.local_bind_port if manager.use_ssh_tunnel else s.port

 2. When we finally enable

 "default_binary_paths": {

 in our test_config, we get more failing tests that look like:

 ==
 FAIL: runTest 
 (pgadmin.tools.restore.tests.test_restore_create_job_unit_test.RestoreCreateJobTest)
 When restore object with option - Miscellaneous
 --
 Traceback (most recent call last):
   File 
 "/Users/pivotal/.pyenv/versions/3.6.5/lib/python3.6/unittest/mock.py", 
 line 1179, in patched
 return func(*args, **keywargs)
   File 
 "/Users/pivotal/workspace/pgadmin4/web/pgadmin/tools/restore/tests/test_restore_create_job_unit_test.py",
  line 295, in runTest
 self.assertEquals(response.status_code, 200)
 AssertionError: 410 != 200

 And

 When restore object with the sections options ... 2018-05-31 12:24:42,988: 
 ERRORpgadmin:illegal environment variable name
 Traceback (most recent call last):
   File 
 "/Users/pivotal/workspace/pgadmin4/web/pgadmin/tools/restore/__init__.py", 
 line 352, in create_restore_job
 manager.export_password_env(p.id)
   File 
 "/Users/pivotal/workspace/pgadmin4/web/pgadmin/utils/driver/psycopg2/server_manager.py",
  line 365, in export_password_env
 os.environ[str(env)] = password
   File "/Users/pivotal/.pyenv/versions/3.6.5/lib/python3.6/os.py", line 
 675, in __setitem__
 self.putenv(key, value)
 ValueError: illegal environment variable name
 FAIL

 ​

 Sincerely,

 Victoria && Anthony

 On Thu, May 31, 2018 at 1:16 AM Khushboo Vashi <
 khushboo.va...@enterprisedb.com> wrote:

> Hi,
>
> Please find the attached updated patch with the fixes.
> The test cases were only failing on MAC not on Linux.
>
> Thanks,
> Khushboo
>
> On Wed, May 30, 2018 at 10:13 AM, Khushboo Vashi <
> khushboo.va...@enterprisedb.com> wrote:
>
>>
>>
>> On Wed, May 30, 2018 at 1:05 AM, Dave Page  wrote:
>>
>>> Hi
>>>
>>> On Mon, May 28, 2018 at 8:09 AM, Khushboo Vashi <
>>> khushboo.va...@enterprisedb.com> wrote:
>>>
 Hi,

 please find the attached updated patch for the test cases of
 Backup, Restore and Maintenance modules which includes:

 1. Unit test cases
 2. End to end regression test cases
 3. Feature test cases

>>>
>>> Thanks. I've yet to be able to run the feature tests successfully.
>>> Here's what I've found so far:
>>>
>>> 1) DEFAULT_BINARY_PATHS should be default_binary_paths in the JSON
>>> config file.
>>>
>>> Will do.
>>
>>> 2) I've hit screensize related issues:
>>>
>>>
>>> ==
>>>

Re: [pgAdmin4][RM#3289] Can't query SQL_ASCII database.

[Aditya Toshniwal]

Hi Hackers,

Please ignore previous patch. Fixed some linter issues. PFA updated patch.

Thanks and Regards,
Aditya Toshniwal
Software Engineer | EnterpriseDB Software Solutions | Pune
"Don't Complain about Heat, Plant a tree"

On Mon, Jun 4, 2018 at 10:53 AM, Aditya Toshniwal <
aditya.toshni...@enterprisedb.com> wrote:

> Hi Hackers,
>
> PFA the updated patch on this. I have tried to add test cases to check for
> different encoding database. In the test run, it will create a database,
> fire a query for a string, check if we get the output and drops the
> database.
> Kindly review.
>
> Thanks and Regards,
> Aditya Toshniwal
> Software Engineer | EnterpriseDB Software Solutions | Pune
> "Don't Complain about Heat, Plant a tree"
>
> On Thu, May 31, 2018 at 6:42 PM, Dave Page  wrote:
>
>> Hi
>>
>> On Thu, May 31, 2018 at 1:20 AM, Aditya Toshniwal <
>> aditya.toshni...@enterprisedb.com> wrote:
>>
>>> Hi Victoria/Joao,
>>>
>>> On Thu, May 31, 2018 at 2:06 AM, Joao De Almeida Pereira <
>>> jdealmeidapere...@pivotal.io> wrote:
>>>
 Hello Aditya,

 It looks ok and it passes CI.

 We have some recommendations:
 - These look like 2 different changes so they should be in separated
 commits

>>>
>>> If you are talking of set client_encoding, then its not a bug. Its a
>>> choice given to Postgres DB user to change the encoding of the characters.
>>> Postgres will translate characters from Server Encoding to Client Encoding,
>>> and will throw error like mentioned previously. This link will help better
>>> - https://www.postgresql.org/docs/10/static/multibyte.html
>>> The actual bug was, even after changing the client encoding to
>>> SQL_ASCII, pgAdmin4 was not able to show the output as it was failing in
>>> encoding by psycopg2. The patch is for resolving that.
>>>
>>>
 - Do we have test coverage for the bug that you are talking about? If
 not we should, to ensure this problem does not happen again in a future
 change.

>>>
>>> It is not possible adding test cases for encoding related stuff, as
>>> Postgres support a lot many different types of encoding and conversions
>>> (refer above link)
>>>
>>
>> I was going to ask the same thing. Per https://www.postgresql.org
>> /docs/10/static/multibyte.html#id-1.6.10.5.7, every characterset except
>> SQL_ASCII can be converted to UTF-8, so we only need to test that UTF-8 and
>> some other charactersets besides SQL_ASCII work, and then separately that
>> SQL_ASCII with characters known not to be in UTF-8 work.
>>
>>
>>>
 Thanks
 Victoria && Joao

 On Wed, May 30, 2018 at 3:06 AM Aditya Toshniwal <
 aditya.toshni...@enterprisedb.com> wrote:

> Hi Hackers,
>
> PFA updated patch after all the permutations, combinations for
> encoding for SQL_ASCII database.  Also fixed a small glitch for sql
> editor connection status check.
>
> Please note, ERROR: invalid byte sequence for encoding "UTF8": 0xe9
> 0x73 is a Postgres DB error and not pgAdmin4 error.
>
> 
>
> You need to change client_encoding to the appropriate. After changing
> client_encoding using command - set client_encoding='XYZ', it will give 
> not
> give error.
>
> 
>
> Thanks and Regards,
> Aditya Toshniwal
> Software Engineer | EnterpriseDB Software Solutions | Pune
> "Don't Complain about Heat, Plant a tree"
>
> On Wed, May 23, 2018 at 10:13 AM, Aditya Toshniwal <
> aditya.toshni...@enterprisedb.com> wrote:
>
>> Thank you Victoria, Anthony.
>>
>> Thanks and Regards,
>> Aditya Toshniwal
>> Software Engineer | EnterpriseDB Software Solutions | Pune
>> "Don't Complain about Heat, Plant a tree"
>>
>> On Tue, May 22, 2018 at 7:15 PM, Victoria Henry 
>> wrote:
>>
>>> Hi Aditya,
>>>
>>> We made a minor change to make the patch so the python linter can
>>> pass.  Attached is the change we made.
>>> Everything else looks good.
>>>
>>> Sincerely,
>>>
>>> Victoria & Anthony
>>>
>>> On Tue, May 22, 2018 at 4:46 AM Aditya Toshniwal <
>>> aditya.toshni...@enterprisedb.com> wrote:
>>>
 Hi,

 PFA updated patch. Linter issues are fixed ( we dont have any
 linter setup for python :-( )
 Regarding test cases, they run successfully on my system and the
 reason it failed for pivotal is timeout exception. I am sorry I can't 
 help
 with that.

 Traceback (most recent call last):
   File 
 "/tmp/build/a453582b/pgadmin-repo/web/pgadmin/feature_tests/keyboard_shortcut_test.py",
 line 52, in runTest
 self._check_shortcuts()
   File 
 "/tmp/build/a453582b/pgadmin-repo/web/pgadmin/feature_tests/keyboard_shortcut_test.py",
 line 77, in _check_shortcuts
 ") and contains(@class, 'open')]")
   File 

Re: [pgAdmin4][RM#3307]User interface unable to connect to database running on port range 1-1024

[Aditya Toshniwal]

Missed the attachment. Apologies :(

PFA.

Thanks and Regards,
Aditya Toshniwal
Software Engineer | EnterpriseDB Software Solutions | Pune
"Don't Complain about Heat, Plant a tree"

On Mon, Jun 4, 2018 at 3:12 PM, Aditya Toshniwal <
aditya.toshni...@enterprisedb.com> wrote:

> Hi Hackers,
>
> I missed one part here. There is a constraint on port in in sqlite config
> database also, and is not allowing ports below 1024.
> PFA patch - part2 which includes constraint change and migration script
> for the db file.
> Kindly review.
>
> Thanks and Regards,
> Aditya Toshniwal
> Software Engineer | EnterpriseDB Software Solutions | Pune
> "Don't Complain about Heat, Plant a tree"
>
> On Fri, May 18, 2018 at 3:43 PM, Dave Page  wrote:
>
>> Thanks, applied.
>>
>> On Wed, May 16, 2018 at 2:34 PM, Victoria Henry 
>> wrote:
>>
>>> Hi Aditya,
>>>
>>> Looks good to us!
>>>
>>> Sincerely,
>>>
>>> Anthony & Victoria
>>>
>>>
>>> On Wed, May 16, 2018 at 7:24 AM Aditya Toshniwal <
>>> aditya.toshni...@enterprisedb.com> wrote:
>>>
 Hi Hackers,

 PFA patch to allow server port number below 1024 till 1 in Create
 Server dialog.

 Thanks and Regards,
 Aditya Toshniwal
 Software Engineer | EnterpriseDB Software Solutions | Pune
 "Don't Complain about Heat, Plant a tree"

>>>
>>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>
>
diff --git a/web/migrations/versions/7c56ea250085_.py b/web/migrations/versions/7c56ea250085_.py
new file mode 100644
index 000..24af349
--- /dev/null
+++ b/web/migrations/versions/7c56ea250085_.py
@@ -0,0 +1,90 @@
+
+"""Change server port constraint to allow port below 1024 RM#3307
+
+Revision ID: 7c56ea250085
+Revises: a68b374fe373
+Create Date: 2018-06-04 14:23:31.472645
+
+"""
+from alembic import op
+import sqlalchemy as sa
+
+from pgadmin.model import db
+
+# revision identifiers, used by Alembic.
+revision = '7c56ea250085'
+down_revision = 'a68b374fe373'
+branch_labels = None
+depends_on = None
+
+
+def upgrade():
+# To Save previous data
+db.engine.execute("ALTER TABLE server RENAME TO server_old")
+
+# Create table with new constraint definition
+db.engine.execute("""
+CREATE TABLE server (
+id	INTEGER NOT NULL,
+user_id	INTEGER NOT NULL,
+servergroup_id	INTEGER NOT NULL,
+name	VARCHAR(128) NOT NULL,
+host	VARCHAR(128),
+port	INTEGER NOT NULL CHECK(port >= 1 AND port <= 65534),
+maintenance_db	VARCHAR(64),
+username	VARCHAR(64) NOT NULL,
+password	VARCHAR(64),
+role	VARCHAR(64),
+ssl_mode	VARCHAR(16) NOT NULL CHECK(ssl_mode IN
+( 'allow' , 'prefer' , 'require' , 'disable' ,
+  'verify-ca' , 'verify-full' )
+),
+comment	VARCHAR(1024),
+discovery_id	VARCHAR(128),
+hostaddr	TEXT(1024),
+db_res	TEXT,
+passfile	TEXT,
+sslcert	TEXT,
+sslkey	TEXT,
+sslrootcert	TEXT,
+sslcrl	TEXT,
+sslcompression	INTEGER DEFAULT 0,
+bgcolor TEXT(10),
+fgcolor TEXT(10),
+service TEXT,
+use_ssh_tunnel INTEGER DEFAULT 0,
+tunnel_host TEXT,
+tunnel_port TEXT,
+tunnel_username TEXT,
+tunnel_authentication INTEGER DEFAULT 0,
+tunnel_identity_file TEXT,
+PRIMARY KEY(id),
+FOREIGN KEY(user_id) REFERENCES user(id),
+FOREIGN KEY(servergroup_id) REFERENCES servergroup(id)
+)
+""")
+
+# Copy old data again into table
+db.engine.execute("""
+INSERT INTO server (
+id, user_id, servergroup_id, name, host, port, maintenance_db,
+username, password, role, ssl_mode, comment, discovery_id, hostaddr,
+db_res, passfile, sslcert, sslkey, sslrootcert, sslcrl,
+sslcompression, bgcolor, fgcolor, service, use_ssh_tunnel,
+tunnel_host, tunnel_port, tunnel_username, tunnel_authentication,
+tunnel_identity_file
+
+) SELECT
+id, user_id, servergroup_id, name, host, port, maintenance_db,
+username, password, role, ssl_mode, comment, discovery_id, hostaddr,
+db_res, passfile, sslcert, sslkey, sslrootcert, sslcrl,
+sslcompression, bgcolor, fgcolor, service, use_ssh_tunnel,
+tunnel_host, tunnel_port, tunnel_username, tunnel_authentication,
+tunnel_identity_file
+FROM server_old""")
+
+# Remove old data
+db.engine.execute("DROP TABLE server_old")
+
+def downgrade():
+pass
diff --git a/web/pgadmin/model/__init__.py b/web/pgadmin/model/__init__.py
index c3171ce..df88116 100644
--- a/web/pgadmin/model/__init__.py
+++ b/web/pgadmin/model/__init__.py
@@ -29,7 +29,7 @@ from flask_sqlalchemy import