[GitHub] madlib pull request #337: Madpack: Add UDO and UDOC automation
Github user asfgit closed the pull request at: https://github.com/apache/madlib/pull/337 ---
[GitHub] madlib pull request #337: Madpack: Add UDO and UDOC automation
Github user jingyimei commented on a diff in the pull request:
https://github.com/apache/madlib/pull/337#discussion_r232721653
--- Diff: src/madpack/diff_udo.sql ---
@@ -0,0 +1,81 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one
+-- or more contributor license agreements. See the NOTICE file
+-- distributed with this work for additional information
+-- regarding copyright ownership. The ASF licenses this file
+-- to you under the Apache License, Version 2.0 (the
+-- "License"); you may not use this file except in compliance
+-- with the License. You may obtain a copy of the License at
+
+-- http://www.apache.org/licenses/LICENSE-2.0
+
+-- Unless required by applicable law or agreed to in writing,
+-- software distributed under the License is distributed on an
+-- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+-- KIND, either express or implied. See the License for the
+-- specific language governing permissions and limitations
+-- under the License.
+--
+
+SET client_min_messages to ERROR;
+\x on
+
+CREATE OR REPLACE FUNCTION filter_schema(argstr text, schema_name text)
+RETURNS text AS $$
+if argstr is None:
+return "NULL"
+return argstr.replace(schema_name + ".", '')
+$$ LANGUAGE plpythonu;
+
+CREATE OR REPLACE FUNCTION alter_schema(argstr text, schema_name text)
+RETURNS text AS $$
+if argstr is None:
+return "NULL"
+return argstr.replace(schema_name + ".", 'schema_madlib.')
+$$ LANGUAGE plpythonu;
+
+
+CREATE OR REPLACE FUNCTION get_udos(table_name text, schema_name text,
+ type_filter text)
+RETURNS VOID AS
+$$
+import plpy
+
+plpy.execute("""
+create table {table_name} AS
+SELECT *
+FROM (
+SELECT n.nspname AS "Schema",
+ o.oprname AS name,
+ filter_schema(o.oprcode::text, '{schema_name}') AS
oprcode,
+ alter_schema(pg_catalog.format_type(o.oprleft,
NULL), '{schema_name}') AS oprleft,
+ alter_schema(pg_catalog.format_type(o.oprright,
NULL), '{schema_name}') AS oprright,
+ alter_schema(pg_catalog.format_type(o.oprresult,
NULL), '{schema_name}') AS rettype
+FROM pg_catalog.pg_operator o
+LEFT JOIN pg_catalog.pg_namespace n ON n.oid =
o.oprnamespace
+WHERE n.nspname OPERATOR(pg_catalog.~) '^({schema_name})$'
--- End diff --
Got it. Thanks for the explanation!
---
[GitHub] madlib pull request #337: Madpack: Add UDO and UDOC automation
Github user orhankislal commented on a diff in the pull request:
https://github.com/apache/madlib/pull/337#discussion_r232279216
--- Diff: src/madpack/create_changelist.py ---
@@ -237,6 +325,13 @@
print "Something went wrong! The changelist might be wrong/corrupted."
raise
finally:
-os.system("rm -f /tmp/madlib_tmp_nm.txt /tmp/madlib_tmp_udf.txt "
- "/tmp/madlib_tmp_udt.txt /tmp/madlib_tmp_cl.yaml "
--- End diff --
Nice catch, it should still be removed.
---
[GitHub] madlib pull request #337: Madpack: Add UDO and UDOC automation
Github user jingyimei commented on a diff in the pull request:
https://github.com/apache/madlib/pull/337#discussion_r231972206
--- Diff: src/madpack/diff_udo.sql ---
@@ -0,0 +1,81 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one
+-- or more contributor license agreements. See the NOTICE file
+-- distributed with this work for additional information
+-- regarding copyright ownership. The ASF licenses this file
+-- to you under the Apache License, Version 2.0 (the
+-- "License"); you may not use this file except in compliance
+-- with the License. You may obtain a copy of the License at
+
+-- http://www.apache.org/licenses/LICENSE-2.0
+
+-- Unless required by applicable law or agreed to in writing,
+-- software distributed under the License is distributed on an
+-- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+-- KIND, either express or implied. See the License for the
+-- specific language governing permissions and limitations
+-- under the License.
+--
+
+SET client_min_messages to ERROR;
+\x on
+
+CREATE OR REPLACE FUNCTION filter_schema(argstr text, schema_name text)
+RETURNS text AS $$
+if argstr is None:
+return "NULL"
+return argstr.replace(schema_name + ".", '')
+$$ LANGUAGE plpythonu;
+
+CREATE OR REPLACE FUNCTION alter_schema(argstr text, schema_name text)
+RETURNS text AS $$
+if argstr is None:
+return "NULL"
+return argstr.replace(schema_name + ".", 'schema_madlib.')
+$$ LANGUAGE plpythonu;
+
+
+CREATE OR REPLACE FUNCTION get_udos(table_name text, schema_name text,
+ type_filter text)
+RETURNS VOID AS
+$$
+import plpy
+
+plpy.execute("""
+create table {table_name} AS
+SELECT *
+FROM (
+SELECT n.nspname AS "Schema",
+ o.oprname AS name,
+ filter_schema(o.oprcode::text, '{schema_name}') AS
oprcode,
+ alter_schema(pg_catalog.format_type(o.oprleft,
NULL), '{schema_name}') AS oprleft,
+ alter_schema(pg_catalog.format_type(o.oprright,
NULL), '{schema_name}') AS oprright,
+ alter_schema(pg_catalog.format_type(o.oprresult,
NULL), '{schema_name}') AS rettype
+FROM pg_catalog.pg_operator o
+LEFT JOIN pg_catalog.pg_namespace n ON n.oid =
o.oprnamespace
+WHERE n.nspname OPERATOR(pg_catalog.~) '^({schema_name})$'
--- End diff --
can you please explain this where clause a bit? I am a bit confused about
the 'OPERATOR' part
---
[GitHub] madlib pull request #337: Madpack: Add UDO and UDOC automation
Github user jingyimei commented on a diff in the pull request:
https://github.com/apache/madlib/pull/337#discussion_r231953942
--- Diff: src/madpack/create_changelist.py ---
@@ -237,6 +325,13 @@
print "Something went wrong! The changelist might be wrong/corrupted."
raise
finally:
-os.system("rm -f /tmp/madlib_tmp_nm.txt /tmp/madlib_tmp_udf.txt "
- "/tmp/madlib_tmp_udt.txt /tmp/madlib_tmp_cl.yaml "
--- End diff --
what is this madlib_tmp_cl.yaml file here? why we don't remove it now
---
[GitHub] madlib pull request #337: Madpack: Add UDO and UDOC automation
GitHub user orhankislal opened a pull request: https://github.com/apache/madlib/pull/337 Madpack: Add UDO and UDOC automation JIRA: MADLIB-1281 - Add scripts for detecting changed/dropped UDOs and UDOCs. - Expand the create_changelist.py file to consume these scripts and create changelists with these fields filled if necessary. - Fix the update_util.py to use the correct dictionary key. - Add drop operator class command to the svac.sql_in to make sure the old class is removed before creating the updated one. You can merge this pull request into a Git repository by running: $ git pull https://github.com/madlib/madlib madpack/complete-changelist Alternatively you can review and apply these changes as the patch at: https://github.com/apache/madlib/pull/337.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #337 commit 09c3bb2e55417309a45f0729f370920273be40b4 Author: Orhan Kislal Date: 2018-10-24T12:55:34Z Madpack: Add UDO and UDOC automation JIRA: MADLIB-1281 - Add scripts for detecting changed/dropped UDOs and UDOCs. - Expand the create_changelist.py file to consume these scripts and create changelists with these fields filled if necessary. - Fix the update_util.py to use the correct dictionary key. - Add drop operator class command to the svac.sql_in to make sure the old class is removed before creating the updated one. ---
