Jira (PDB-3322) PuppetDB fails to update catalogs when sensitive parameters are added

[Charlie Sharpsteen (JIRA)]

Title: Message Title

Charlie Sharpsteen created an issue   PuppetDB / PDB-3322 PuppetDB fails to update catalogs when sensitive parameters are added Issue Type: Bug Affects Versions: PDB 4.2.3 Assignee: Unassigned Created: 2017/03/07 8:11 AM Priority: Normal Reporter: Charlie Sharpsteen When sensitive data is added to a resource, the resulting catalog contains a sensitive_parameters key. PuppetDB tries to work this new into an UPDATE operation on the catalog_resources table, which fails during statement preparation with a 'Can't infer the SQL type to use for an instance of clojure.lang.PersistentVector' error. No errors occur if the resource is initially stored with sensitive data. Reproduction Case Install PE 2016.4.3. Add a user resource to the default node in /etc/puppetlabs/code/environments/production/manifests/site.pp: user {'AzureDiamond': ensure => present, password => 'hunter2', } Run puppet agent -t to compile a catalog and enter the resource into PuppetDB. Update the resource to mark the password as sensitive: user {'AzureDiamond': ensure => present, password => Sensitive.new('hunter2'), } Run puppet agent -t. Outcome The second agent run completes successfully, but /var/log/puppetlabs/puppetdb/puppetdb.log shows an error during catalog storage: 2017-03-07 15:29:15,702 ERROR [p.p.mq-listener] [57cc2d5b-5334-47a1-8426-c4f1b78db715] [replace catalog] Retrying after attempt 0 for pe-201651-master.puppetdebug.vlan, due to: org.postgresql.util.PSQLException: Can't infer the SQL type to use for an instance of clojure.lang.PersistentVector. Use setObject() with an explicit Types value to specify the type to use. org.postgresql.util.PSQLException: Can't infer the SQL type to use for an instance of clojure.lang.PersistentVector. Use setObject() with an explicit Types value to specify the type to use. at org.postgresql.jdbc.PgPreparedStatement.setObject(PgPreparedStatement.java:1039) at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.setObject(HikariProxyPreparedStatement.java) at clojure.java.jdbc$eval21273$fn__21274.invokePrim(jdbc.clj:341) at clojure.java.jdbc$eval21273$fn__21274.invoke(jdbc.clj) at clojure.java.jdbc$eval21252$fn__21253$G__21243__21262.invoke(jdbc.clj:328) at clojure.java.jdbc$set_parameters$fn__21349.invoke(jdbc.clj:478) at clojure.core$map_indexed$mapi__7050$fn__7051.invoke(core.clj:7024) at clojure.lang.LazySeq.sval(LazySeq.java:40) at clojure.lang.LazySeq.seq(LazySeq.java:49) at clojure.lang.RT.seq(RT.java:521) at clojure.core$seq__4357.invokeStatic(core.clj:137) at clojure.core$dorun.invokeStatic(core.clj:3024) at clojure.core$dorun.invoke(core.clj:3024) at clojure.java.jdbc$set_parameters.invokeStatic(jdbc.clj:477) at clojure.java.jdbc$set_parameters.invoke(jdbc.clj:474) at clojure.java.jdbc$db_do_execute_prepared_statement.invokeStatic(jdbc.clj:760) at clojure.java.jdbc$db_do_execute_prepared_statement.invoke(jdbc.clj:748) at clojure.java.jdbc$db_do_prepared.invokeStatic(jdbc.clj:786) at clojure.java.jdbc$db_do_prepared.doInvoke(jdbc.clj:770) at clojure.lang.RestFn.invoke(RestFn.java:464) at clojure.java.jdbc$execute_BANG_$execute_helper__21451.invoke(jdbc.clj:891) at clojure.java.jdbc$execute_BANG_.invokeStatic(jdbc.clj:894) at clojure.java.jdbc$execute_BANG_.doInvoke(jdbc.clj:875) at clojure.lang.RestFn.invoke(RestFn.java:464) at clojure.java.jdbc$update_BANG_.invokeStatic(jdbc.clj:1077) at clojure.java.jdbc$update_BANG_.doInvoke(jdbc.clj:1066) at clojure.lang.RestFn.invoke(RestFn.java:470) at clojure.lang.AFn.applyToHelper(AFn.java:165) at clojure.lang.RestFn.applyTo(RestFn.java:132) at clojure.core$apply.invokeStatic(core.clj:654) at clojure.core$apply.doInvoke(core.clj:641) at clojure.lang.RestFn.invoke(RestFn.java:533) at puppetlabs.puppetdb.jdbc$update_BANG_.invokeStatic(jdbc.clj:72) at puppetlabs.puppetdb.jdbc$update_BANG_.doInvoke(jdbc.clj:66) at clojure.lang.RestFn.invoke(RestFn.java:445) at puppetlabs.puppetdb.scf.storage$eval28606$update_catalog_resources_BANG___28611$fn__28612$fn__28613.invoke(storage.clj:583) at puppetlabs.puppetdb.utils$eval6348$diff_fn__6353$fn__6354.invoke(utils.clj:65) at puppetlabs.puppetdb.utils$eval6348$diff_fn__6353.invoke(utils.clj:54) at puppetlabs.puppetdb.scf.storage$eval28646$add_resources_BANG___28651$fn__28652$fn__28653.invoke(storage.clj:603) at clojure.java.jdbc$db_transaction_STAR_.invokeStatic(jdbc.clj:620) at clojure.java.jdbc$db_transaction_STAR_.doInvoke(jdbc.clj:568) at clojure.lang.RestFn.invoke(RestFn.java:425) at puppetlabs.puppetdb.scf.storage$eval28646$add_resources_BANG___28651$fn__28652.invoke(storage.clj:601) at puppetlabs.puppetdb.scf.storage$eval28646$add_resources_BANG___28651.invoke(storage.clj:594) at puppetlabs.puppetdb.scf.storage$eval28847$update_catalog_associations_BANG___28852$fn__28856$fn__28858.invoke(storage.clj:713) at puppetlabs.puppetdb.scf.storage.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) at com.codahale.metrics.Timer.time(Timer.java:101) at puppetlabs.puppetdb.scf.storage$eval28847$update_catalog_associations_BANG___28852$fn__28856.invoke(storage.clj:712) at puppetlabs.puppetdb.scf.storage$eval28847$update_catalog_associations_BANG___28852.invoke(storage.clj:707) at puppetlabs.puppetdb.scf.storage$eval28885$replace_existing_catalog__28890$fn__28891$fn__28892.invoke(storage.clj:731) at puppetlabs.puppetdb.scf.storage.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) at com.codahale.metrics.Timer.time(Timer.java:101) at puppetlabs.puppetdb.scf.storage$eval28885$replace_existing_catalog__28890$fn__28891.invoke(storage.clj:729) at puppetlabs.puppetdb.scf.storage$eval28885$replace_existing_catalog__28890.invoke(storage.clj:717) at puppetlabs.puppetdb.scf.storage$eval28946$replace_catalog_BANG___28955$fn__28961$fn__28963$fn__28964.invoke(storage.clj:773) at clojure.java.jdbc$db_transaction_STAR_.invokeStatic(jdbc.clj:620) at clojure.java.jdbc$db_transaction_STAR_.doInvoke(jdbc.clj:568) at clojure.lang.RestFn.invoke(RestFn.java:425) at puppetlabs.puppetdb.scf.storage$eval28946$replace_catalog_BANG___28955$fn__28961$fn__28963.invoke(storage.clj:753) at puppetlabs.puppetdb.scf.storage.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) at com.codahale.metrics.Timer.time(Timer.java:101) at puppetlabs.puppetdb.scf.storage$eval28946$replace_catalog_BANG___28955$fn__28961.invoke(storage.clj:752) at puppetlabs.puppetdb.scf.storage$eval28946$replace_catalog_BANG___28955.invoke(storage.clj:745) at puppetlabs.puppetdb.command$replace_catalog_STAR_$fn__42268.invoke(command.clj:259) at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn$fn__21948$fn__21949.invoke(jdbc.clj:308) at clojure.java.jdbc$db_transaction_STAR_.invokeStatic(jdbc.clj:595) at clojure.java.jdbc$db_transaction_STAR_.doInvoke(jdbc.clj:568) at clojure.lang.RestFn.invoke(RestFn.java:464) at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn$fn__21948.invoke(jdbc.clj:307) at puppetlabs.puppetdb.jdbc$eval21922$retry_sql_STAR___21927$fn__21928$fn__21929.invoke(jdbc.clj:285) at puppetlabs.puppetdb.jdbc$eval21922$retry_sql_STAR___21927$fn__21928.invoke(jdbc.clj:284) at puppetlabs.puppetdb.jdbc$eval21922$retry_sql_STAR___21927.invoke(jdbc.clj:275) at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn.invokeStatic(jdbc.clj:305) at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn.invoke(jdbc.clj:300) at puppetlabs.puppetdb.command$replace_catalog_STAR_.invokeStatic(command.clj:257) at puppetlabs.puppetdb.command$replace_catalog_STAR_.invoke(command.clj:253) at puppetlabs.puppetdb.command$replace_catalog.invokeStatic(command.clj:268) at puppetlabs.puppetdb.command$replace_catalog.invoke(command.clj:262) at puppetlabs.puppetdb.command$process_command_BANG_.invokeStatic(command.clj:373) at puppetlabs.puppetdb.command$process_command_BANG_.invoke(command.clj:368) at puppetlabs.puppetdb.command$process_command_and_respond_BANG_$fn__42465.invoke(command.clj:435) at puppetlabs.puppetdb.command$call_with_quick_retry$fn__42460.invoke(command.clj:421) at puppetlabs.puppetdb.command$call_with_quick_retry.invokeStatic(command.clj:420) at puppetlabs.puppetdb.command$call_with_quick_retry.invoke(command.clj:418) at puppetlabs.puppetdb.command$process_command_and_respond_BANG_.invokeStatic(command.clj:433) at puppetlabs.puppetdb.command$process_command_and_respond_BANG_.invoke(command.clj:431) at puppetlabs.puppetdb.command$reify__42469$service_fnk__11430__auto___positional$reify__42480$fn__42483.invoke(command.clj:471) at puppetlabs.puppetdb.mq_listener$reify__42717$service_fnk__11430__auto___positional$reify__42727.process_message(mq_listener.clj:399) at puppetlabs.puppetdb.mq_listener$reify__42717$service_fnk__11430__auto___positional$reify__42727$process_msg__42728.invoke(mq_listener.clj:367) at puppetlabs.puppetdb.mq_listener$wrap_with_discard$fn__42545$fn__42547.invoke(mq_listener.clj:228) at puppetlabs.puppetdb.mq_listener.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) at com.codahale.metrics.Timer.time(Timer.java:101) at puppetlabs.puppetdb.mq_listener$wrap_with_discard$fn__42545.invoke(mq_listener.clj:227) at puppetlabs.puppetdb.mq_listener$wrap_with_exception_handling$fn__42533$fn__42535.invoke(mq_listener.clj:182) at puppetlabs.puppetdb.mq_listener.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) at com.codahale.metrics.Timer.time(Timer.java:101) at puppetlabs.puppetdb.mq_listener$wrap_with_exception_handling$fn__42533.invoke(mq_listener.clj:181) at puppetlabs.puppetdb.mq_listener$wrap_with_command_parser$fn__42541.invoke(mq_listener.clj:204) at puppetlabs.puppetdb.mq_listener$wrap_with_meter$fn__42525.invoke(mq_listener.clj:142) at puppetlabs.puppetdb.mq_listener$wrap_with_thread_name$fn__42552.invoke(mq_listener.clj:243) at puppetlabs.puppetdb.mq_listener$start_receiver$reify__42710.onMessage(mq_listener.clj:347) at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1401) at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131) at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202) at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133) at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Dropping a tracepoint into clojure.java.jdbc$eval21273$fn__21274.invokePrim(jdbc.clj:341) shows that the following UPDATE statement is being prepared and that the error occurs when parameter 1 is being set to ["password"]: [#object[com.zaxxer.hikari.pool.HikariProxyPreparedStatement 0x242441f1 "HikariProxyPreparedStatement@606355953 wrapping UPDATE catalog_resources SET sensitive_parameters = ? WHERE certname_id = ? and type = ? and title = ?"] 1 ["password"]] Expected Outcome PuppetDB should accept catalogs updates where resource parameters have been marked as sensitive. Add Comment   This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.