Eileen has uploaded a new change for review. https://gerrit.wikimedia.org/r/293891
Change subject: WMF patches over latest master. ...................................................................... WMF patches over latest master. Includes all our not-yet-merged patches I wanted to push the branches this is compiled from to gerrit but gerrit was less keen. Patches reflect 2 patches in packages: https://github.com/civicrm/civicrm-packages/compare/master...eileenmcnaughton:4.7.8?expand=1 And 25 patches here: (some of which are still in QA for 4.7.9) https://github.com/civicrm/civicrm-core/compare/master...eileenmcnaughton:wmf-4.7.8?expand=1 @eileenmcnaughton wmf: CRM-17157 allow more than 2 decimal places in currency … 2ca179f @eileenmcnaughton wmf: CRM-10700 use DELETE FROM instead of TRUNCATE … 594acf1 @eileenmcnaughton wmf: CRM-17158 block empty searches unless deliberate … f7c3324 @eileenmcnaughton wmf-sunset patch add extra columns to contribution recur tab … ddfaf47 @eileenmcnaughton wmf-sunset patch skip process greetings on contact create … 012052b @adamwight Merge from master :Include custom fields in default export … 9bdd572 @adamwight Don't create SQL function at runtime … 9562633 eileen Return more rows per query and use unbuffered query to manage the memory … 4bc658b eileen Increase time out while doing exports … c069fd9 (QA 4.7.9) eileen needs review CRM-17154 wmf-gr report actions improvement … c3dc5b3 @eileenmcnaughton Deployment file adds: gitreview, settings.location, readme, submodules … 73af99e (does not alter deployed) Add script for compiling deployment repo … 02bab90 eileen Hack for running GenCode … ba5b1ba (QA 4.7.9) eileen CRM-18681 respect api permissions flag on batch merge job 8d830e9 (QA 4.7.9) eileen Cleanup a couple of unused vars 7534660 (QA 4.7.9) eileen Make the changes John suggested last time, so the tests don't break E… … 2b55947 (QA 4.7.9) eileen CRM-18685 Batch merge - custom data not treated as a conflict - resul… … d335217 (QA 4.7.9) eileen CRM-18674 fix date merging for view_only custom fields 7ca2b85 (QA 4.7.9) eileen CRM-18761 fix for dodgey date revert 9e2d509 eileen quick hack to group safety … 3ec11c3 (QA 4.7.9) eileen To be upstreamed - further custom data perms issue … 05f6d29 eileen Fix another slow point … 97b2340 (perf - need to fix better upstream) @eileenmcnaughton Revert "add smartGroup title in select clause" … e576613 @eileenmcnaughton eileenmcnaughton wmf-sunset-patch alter exception behaviour … 84a3a6c Change-Id: I37c9a5af150e0d4ae91841f50bc24e488fc729c3 --- M CRM/ACL/BAO/Cache.php M CRM/Contact/BAO/Contact.php M CRM/Contact/BAO/GroupContactCache.php M CRM/Contact/BAO/Query.php M CRM/Contribute/BAO/ContributionRecur.php M CRM/Contribute/Form/Search.php M CRM/Core/BAO/PrevNextCache.php M CRM/Core/Config.php M CRM/Core/DAO.php M CRM/Core/Error.php M CRM/Core/Form.php M CRM/Core/Menu.php M CRM/Core/OptionValue.php M CRM/Core/Session.php M CRM/Core/TableHierarchy.php M CRM/Core/TemporaryErrorScope.php M CRM/Dedupe/BAO/Rule.php M CRM/Dedupe/BAO/RuleGroup.php M CRM/Dedupe/Finder.php M CRM/Dedupe/Merger.php M CRM/Export/BAO/Export.php M CRM/Logging/Reverter.php M CRM/Report/Form.php M CRM/Report/Form/Contribute/Detail.php M CRM/Report/Form/Instance.php M CRM/Utils/Rule.php M api/v3/Job.php M packages/DB/common.php M packages/DB/mysql.php A settings_location.php M templates/CRM/Report/Form.tpl M templates/CRM/Report/Form/Actions.tpl A templates/CRM/common/tasks.tpl 33 files changed, 303 insertions(+), 124 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm/civicrm refs/changes/91/293891/1 diff --git a/CRM/ACL/BAO/Cache.php b/CRM/ACL/BAO/Cache.php index 11ac3d3..1cc0ea1 100644 --- a/CRM/ACL/BAO/Cache.php +++ b/CRM/ACL/BAO/Cache.php @@ -158,11 +158,11 @@ // CRM_Core_DAO::singleValueQuery("DELETE FROM civicrm_acl_contact_cache"); // Transaction-safe if (CRM_Core_Transaction::isActive()) { CRM_Core_Transaction::addCallback(CRM_Core_Transaction::PHASE_POST_COMMIT, function () { - CRM_Core_DAO::singleValueQuery("TRUNCATE TABLE civicrm_acl_contact_cache"); + CRM_Core_DAO::singleValueQuery("DELETE FROM civicrm_acl_contact_cache"); }); } else { - CRM_Core_DAO::singleValueQuery("TRUNCATE TABLE civicrm_acl_contact_cache"); + CRM_Core_DAO::singleValueQuery("DELETE FROM civicrm_acl_contact_cache"); } } diff --git a/CRM/Contact/BAO/Contact.php b/CRM/Contact/BAO/Contact.php index 6266a44..8c79dff 100644 --- a/CRM/Contact/BAO/Contact.php +++ b/CRM/Contact/BAO/Contact.php @@ -450,7 +450,7 @@ } // process greetings CRM-4575, cache greetings - self::processGreetings($contact); + // NOT: self::processGreetings($contact); return $contact; } @@ -2253,7 +2253,9 @@ $value, $type, $valueId, - $contactID + $contactID, + FALSE, + FALSE ); } elseif ($key == 'edit') { @@ -3285,8 +3287,11 @@ // Update phone table to populate phone_numeric field if (!$tableName || $tableName == 'civicrm_phone') { // Define stored sql function needed for phones - CRM_Core_DAO::executeQuery(self::DROP_STRIP_FUNCTION_43); - CRM_Core_DAO::executeQuery(self::CREATE_STRIP_FUNCTION_43); + // + // FIXME: WMF bug T117044, we can't create this function at runtime (unknown why not). + // CRM_Core_DAO::executeQuery(self::DROP_STRIP_FUNCTION_43); + // CRM_Core_DAO::executeQuery(self::CREATE_STRIP_FUNCTION_43); + $info[] = array( 'table' => array('civicrm_phone'), 'when' => 'BEFORE', diff --git a/CRM/Contact/BAO/GroupContactCache.php b/CRM/Contact/BAO/GroupContactCache.php index cdb781d..dba6976 100644 --- a/CRM/Contact/BAO/GroupContactCache.php +++ b/CRM/Contact/BAO/GroupContactCache.php @@ -354,7 +354,7 @@ if (!isset($groupID)) { if ($smartGroupCacheTimeout == 0) { $query = " -TRUNCATE civicrm_group_contact_cache +DELETE FROM civicrm_group_contact_cache "; $update = " UPDATE civicrm_group g diff --git a/CRM/Contact/BAO/Query.php b/CRM/Contact/BAO/Query.php index 561ae08..2f4dd44 100644 --- a/CRM/Contact/BAO/Query.php +++ b/CRM/Contact/BAO/Query.php @@ -2960,7 +2960,7 @@ if ($ssClause) { $and = ($op == 'IS NULL') ? 'AND' : 'OR'; - if ($groupClause) { + if ($groupClause && $and != 'OR') { $groupClause = "( ( $groupClause ) $and ( $ssClause ) )"; } else { diff --git a/CRM/Contribute/BAO/ContributionRecur.php b/CRM/Contribute/BAO/ContributionRecur.php index 0221486..e434736 100644 --- a/CRM/Contribute/BAO/ContributionRecur.php +++ b/CRM/Contribute/BAO/ContributionRecur.php @@ -351,6 +351,8 @@ $params[$recurDAO->id]['next_sched_contribution_date'] = $recurDAO->next_sched_contribution_date; $params[$recurDAO->id]['amount'] = $recurDAO->amount; $params[$recurDAO->id]['currency'] = $recurDAO->currency; + $params[$recurDAO->id]['failure_count'] = $recurDAO->failure_count; + $params[$recurDAO->id]['failure_retry_date'] = $recurDAO->failure_retry_date; $params[$recurDAO->id]['frequency_unit'] = $recurDAO->frequency_unit; $params[$recurDAO->id]['frequency_interval'] = $recurDAO->frequency_interval; $params[$recurDAO->id]['installments'] = $recurDAO->installments; diff --git a/CRM/Contribute/Form/Search.php b/CRM/Contribute/Form/Search.php index 866c736..4954884 100644 --- a/CRM/Contribute/Form/Search.php +++ b/CRM/Contribute/Form/Search.php @@ -85,6 +85,13 @@ $this->_limit = CRM_Utils_Request::retrieve('limit', 'Positive', $this); $this->_context = CRM_Utils_Request::retrieve('context', 'String', $this, FALSE, 'search'); + /* + * WMF HACK: "force" causes a search with null criteria. Disable until this is fixed. + */ + if ( $this->_context === "search" && CRM_Utils_Request::retrieve('qfKey', 'String') === NULL ) { + $this->_force = false; + } + $this->assign("context", $this->_context); // get user submitted values diff --git a/CRM/Core/BAO/PrevNextCache.php b/CRM/Core/BAO/PrevNextCache.php index 8cacb0d..91bb827 100644 --- a/CRM/Core/BAO/PrevNextCache.php +++ b/CRM/Core/BAO/PrevNextCache.php @@ -344,11 +344,16 @@ * @param array $criteria * Additional criteria to filter by. * + * @param bool $checkPermissions + * Respect logged in user's permissions. + * * @return bool + * @throws \CRM_Core_Exception + * @throws \CiviCRM_API3_Exception */ - public static function refillCache($rgid = NULL, $gid = NULL, $cacheKeyString = NULL, $criteria = array()) { + public static function refillCache($rgid, $gid, $cacheKeyString, $criteria, $checkPermissions) { if (!$cacheKeyString && $rgid) { - $cacheKeyString = CRM_Dedupe_Merger::getMergeCacheKeyString($rgid, $gid, $criteria); + $cacheKeyString = CRM_Dedupe_Merger::getMergeCacheKeyString($rgid, $gid, $criteria, $checkPermissions); } if (!$cacheKeyString) { @@ -373,7 +378,7 @@ $contacts = civicrm_api3('Contact', 'get', array_merge(array('options' => array('limit' => 0), 'return' => 'id'), $criteria['contact'])); $contactIDs = array_keys($contacts['values']); } - $foundDupes = CRM_Dedupe_Finder::dupes($rgid, $contactIDs); + $foundDupes = CRM_Dedupe_Finder::dupes($rgid, $contactIDs, $checkPermissions); } if (!empty($foundDupes)) { diff --git a/CRM/Core/Config.php b/CRM/Core/Config.php index 3d4d587..68b276a 100644 --- a/CRM/Core/Config.php +++ b/CRM/Core/Config.php @@ -324,13 +324,13 @@ */ public static function clearDBCache() { $queries = array( - 'TRUNCATE TABLE civicrm_acl_cache', - 'TRUNCATE TABLE civicrm_acl_contact_cache', - 'TRUNCATE TABLE civicrm_cache', - 'TRUNCATE TABLE civicrm_prevnext_cache', + 'DELETE FROM civicrm_acl_cache', + 'DELETE FROM civicrm_acl_contact_cache', + 'DELETE FROM civicrm_cache', + 'DELETE FROM civicrm_prevnext_cache', 'UPDATE civicrm_group SET cache_date = NULL', - 'TRUNCATE TABLE civicrm_group_contact_cache', - 'TRUNCATE TABLE civicrm_menu', + 'DELETE FROM civicrm_group_contact_cache', + 'DELETE FROM civicrm_menu', 'UPDATE civicrm_setting SET value = NULL WHERE name="navigation" AND contact_id IS NOT NULL', 'DELETE FROM civicrm_setting WHERE name="modulePaths"', // CRM-10543 ); diff --git a/CRM/Core/DAO.php b/CRM/Core/DAO.php index a168da8..4e942b1 100644 --- a/CRM/Core/DAO.php +++ b/CRM/Core/DAO.php @@ -108,6 +108,10 @@ self::DebugLevel(CIVICRM_DAO_DEBUG); } CRM_Core_DAO::setFactory(new CRM_Contact_DAO_Factory()); + if (!$dsn) { + // Hack for running GenCode without a dsn. + return; + } if (CRM_Utils_Constant::value('CIVICRM_MYSQL_STRICT', CRM_Utils_System::isDevelopment())) { CRM_Core_DAO::executeQuery('SET SESSION sql_mode = STRICT_TRANS_TABLES'); } diff --git a/CRM/Core/Error.php b/CRM/Core/Error.php index 7d488ee..bea5d08 100644 --- a/CRM/Core/Error.php +++ b/CRM/Core/Error.php @@ -95,7 +95,7 @@ /** * If modeException == true, errors are raised as exception instead of returning civicrm_errors */ - public static $modeException = NULL; + public static $modeException = true; /** * Singleton function used to manage this object. @@ -540,7 +540,7 @@ public static function debug_var( $variable_name, $variable, - $print = TRUE, + $print = FALSE, $log = TRUE, $comp = '' ) { diff --git a/CRM/Core/Form.php b/CRM/Core/Form.php index 266f82b..b8bad65 100644 --- a/CRM/Core/Form.php +++ b/CRM/Core/Form.php @@ -2171,6 +2171,31 @@ } /** + * Add actions menu to results form. + * + * @param $tasks + */ + public function addTaskMenu($tasks) { + if (is_array($tasks) && !empty($tasks)) { + $tasks = array('' => ts('Actions')) + $tasks; + $this->add('select', 'task', NULL, $tasks, FALSE, array('class' => 'crm-select2 crm-action-menu huge crm-search-result-actions')); + if (empty($this->_actionButtonName)) { + $this->_actionButtonName = $this->getButtonName('next', 'action'); + } + $this->assign('actionButtonName', $this->_actionButtonName); + $this->add('submit', $this->_actionButtonName, ts('Go'), array('class' => 'hiddenElement crm-search-go-button')); + + // Radio to choose "All items" or "Selected items only" + $selectedRowsRadio = $this->addElement('radio', 'radio_ts', NULL, '', 'ts_sel', array('checked' => 'checked')); + $allRowsRadio = $this->addElement('radio', 'radio_ts', NULL, '', 'ts_all'); + $this->assign('ts_sel_id', $selectedRowsRadio->_attributes['id']); + $this->assign('ts_all_id', $allRowsRadio->_attributes['id']); + + CRM_Core_Resources::singleton()->addScriptFile('civicrm', 'js/crm.searchForm.js', 1, 'html-header'); + } + } + + /** * Set options and attributes for chain select fields based on the controlling field's value */ private function preProcessChainSelectFields() { diff --git a/CRM/Core/Menu.php b/CRM/Core/Menu.php index e100d99..0808258 100644 --- a/CRM/Core/Menu.php +++ b/CRM/Core/Menu.php @@ -275,7 +275,7 @@ public static function store($truncate = TRUE) { // first clean up the db if ($truncate) { - $query = 'TRUNCATE civicrm_menu'; + $query = 'DELETE FROM civicrm_menu'; CRM_Core_DAO::executeQuery($query); } $menuArray = self::items($truncate); diff --git a/CRM/Core/OptionValue.php b/CRM/Core/OptionValue.php index fc96a9b..ac7dbad 100644 --- a/CRM/Core/OptionValue.php +++ b/CRM/Core/OptionValue.php @@ -271,8 +271,7 @@ * @param string $mode * @param string $contactType * - * @return bool - * true if object exists + * @return array */ public static function getFields($mode = '', $contactType = 'Individual') { $key = "$mode $contactType"; diff --git a/CRM/Core/Session.php b/CRM/Core/Session.php index 0cc4155..7cec890 100644 --- a/CRM/Core/Session.php +++ b/CRM/Core/Session.php @@ -564,9 +564,7 @@ /** * Check if session is empty. * - * if so we don't cache stuff that we can get away with, helps proxies like varnish. - * - * @return bool + * if so we don't cache stuff that we can get away with, helps proxies like varnish. * @return bool */ public function isEmpty() { return empty($_SESSION); diff --git a/CRM/Core/TableHierarchy.php b/CRM/Core/TableHierarchy.php index 94c6abe..096a9a7 100644 --- a/CRM/Core/TableHierarchy.php +++ b/CRM/Core/TableHierarchy.php @@ -51,30 +51,29 @@ 'civicrm_openid' => '17', 'civicrm_location_type' => '18', 'civicrm_group_contact' => '19', - 'civicrm_group_contact_cache' => '20', - 'civicrm_group' => '21', - 'civicrm_subscription_history' => '22', - 'civicrm_entity_tag' => '23', - 'civicrm_note' => '24', - 'civicrm_contribution' => '25', - 'civicrm_financial_type' => '26', - 'civicrm_participant' => '27', - 'civicrm_event' => '28', - 'civicrm_worldregion' => '29', - 'civicrm_case_contact' => '30', - 'civicrm_case' => '31', - 'case_relationship' => '32', - 'case_relation_type' => '33', - 'civicrm_activity' => '34', - 'civicrm_mailing_summary' => '35', - 'civicrm_mailing_recipients' => '36', - 'civicrm_mailing' => '37', - 'civicrm_mailing_job' => '38', - 'civicrm_mailing_event_queue' => '39', - 'civicrm_mailing_event_bounce' => '40', - 'civicrm_mailing_event_opened' => '41', - 'civicrm_mailing_event_reply' => '42', - 'civicrm_mailing_event_trackable_url_open' => '43', + 'civicrm_group' => '20', + 'civicrm_subscription_history' => '21', + 'civicrm_entity_tag' => '22', + 'civicrm_note' => '23', + 'civicrm_contribution' => '24', + 'civicrm_financial_type' => '25', + 'civicrm_participant' => '26', + 'civicrm_event' => '27', + 'civicrm_worldregion' => '28', + 'civicrm_case_contact' => '29', + 'civicrm_case' => '30', + 'case_relationship' => '31', + 'case_relation_type' => '32', + 'civicrm_activity' => '33', + 'civicrm_mailing_summary' => '34', + 'civicrm_mailing_recipients' => '35', + 'civicrm_mailing' => '36', + 'civicrm_mailing_job' => '37', + 'civicrm_mailing_event_queue' => '38', + 'civicrm_mailing_event_bounce' => '39', + 'civicrm_mailing_event_opened' => '40', + 'civicrm_mailing_event_reply' => '41', + 'civicrm_mailing_event_trackable_url_open' => '42', ); /** diff --git a/CRM/Core/TemporaryErrorScope.php b/CRM/Core/TemporaryErrorScope.php index 7f78496..e60965c 100644 --- a/CRM/Core/TemporaryErrorScope.php +++ b/CRM/Core/TemporaryErrorScope.php @@ -70,6 +70,13 @@ * Read the active error-handler settings */ public static function getActive() { + if ( !isset( $GLOBALS['_PEAR_default_error_mode'] ) ) { + return array( + '_PEAR_default_error_mode' => null, + '_PEAR_default_error_options' => null, + 'modeException' => CRM_Core_Error::$modeException, + ); + } return array( '_PEAR_default_error_mode' => $GLOBALS['_PEAR_default_error_mode'], '_PEAR_default_error_options' => $GLOBALS['_PEAR_default_error_options'], diff --git a/CRM/Dedupe/BAO/Rule.php b/CRM/Dedupe/BAO/Rule.php index 552dc2b..5af89b8 100644 --- a/CRM/Dedupe/BAO/Rule.php +++ b/CRM/Dedupe/BAO/Rule.php @@ -171,20 +171,22 @@ $where[] = "t1.{$this->rule_field} IS NOT NULL"; $where[] = "t1.{$this->rule_field} <> ''"; } + $query = "SELECT $select FROM $from WHERE " . implode(' AND ', $where); if ($this->contactIds) { $cids = array(); foreach ($this->contactIds as $cid) { $cids[] = CRM_Utils_Type::escape($cid, 'Integer'); } if (count($cids) == 1) { - $where[] = "(t1.$id = {$cids[0]} OR t2.$id = {$cids[0]})"; + $query .= " AND (t1.$id = {$cids[0]} UNION $query AND t2.$id = {$cids[0]})"; } else { - $where[] = "(t1.$id IN (" . implode(',', $cids) . ") OR t2.$id IN (" . implode(',', $cids) . "))"; + $query .= " AND (t1.$id IN (" . implode(',', $cids) . ")) + UNION $query AND (t2.$id IN (" . implode(',', $cids) . "))"; } } - return "SELECT $select FROM $from WHERE " . implode(' AND ', $where); + return $query; } /** diff --git a/CRM/Dedupe/BAO/RuleGroup.php b/CRM/Dedupe/BAO/RuleGroup.php index 56081b4..dafa322 100644 --- a/CRM/Dedupe/BAO/RuleGroup.php +++ b/CRM/Dedupe/BAO/RuleGroup.php @@ -231,6 +231,9 @@ preg_match($patternColumn, $query, $matches); $query = str_replace(' WHERE ', str_replace('column', $matches[1], $dupeCopyJoin), $query); + if (($u = strpos($query, 'UNION')) !== False) { + $query = substr($query, 0, $u); + } } $searchWithinDupes = 1; diff --git a/CRM/Dedupe/Finder.php b/CRM/Dedupe/Finder.php index cb0bc26..38fdf81 100644 --- a/CRM/Dedupe/Finder.php +++ b/CRM/Dedupe/Finder.php @@ -48,10 +48,14 @@ * @param array $cids * Contact ids to limit the search to. * + * @param bool $checkPermissions + * Respect logged in user permissions. + * * @return array - * array of (cid1, cid2, weight) dupe triples + * Array of (cid1, cid2, weight) dupe triples + * @throws \Exception */ - public static function dupes($rgid, $cids = array()) { + public static function dupes($rgid, $cids = array(), $checkPermissions = TRUE) { $rgBao = new CRM_Dedupe_BAO_RuleGroup(); $rgBao->id = $rgid; $rgBao->contactIds = $cids; @@ -61,7 +65,7 @@ $rgBao->fillTable(); $dao = new CRM_Core_DAO(); - $dao->query($rgBao->thresholdQuery()); + $dao->query($rgBao->thresholdQuery($checkPermissions)); $dupes = array(); while ($dao->fetch()) { $dupes[] = array($dao->id1, $dao->id2, $dao->weight); diff --git a/CRM/Dedupe/Merger.php b/CRM/Dedupe/Merger.php index 0bc09a6..5da3c9a 100644 --- a/CRM/Dedupe/Merger.php +++ b/CRM/Dedupe/Merger.php @@ -593,21 +593,24 @@ * @param array $criteria * Criteria to use in the filter. * + * @param bool $checkPermissions + * Respect logged in user permissions. + * * @return array|bool */ - public static function batchMerge($rgid, $gid = NULL, $mode = 'safe', $autoFlip = TRUE, $batchLimit = 1, $isSelected = 2, $criteria = array()) { + public static function batchMerge($rgid, $gid = NULL, $mode = 'safe', $autoFlip = TRUE, $batchLimit = 1, $isSelected = 2, $criteria = array(), $checkPermissions = TRUE) { $redirectForPerformance = ($batchLimit > 1) ? TRUE : FALSE; $reloadCacheIfEmpty = (!$redirectForPerformance && $isSelected == 2); - $dupePairs = self::getDuplicatePairs($rgid, $gid, $reloadCacheIfEmpty, $batchLimit, $isSelected, '', ($mode == 'aggressive'), $criteria); + $dupePairs = self::getDuplicatePairs($rgid, $gid, $reloadCacheIfEmpty, $batchLimit, $isSelected, '', ($mode == 'aggressive'), $criteria, $checkPermissions); $cacheParams = array( - 'cache_key_string' => self::getMergeCacheKeyString($rgid, $gid, $criteria), + 'cache_key_string' => self::getMergeCacheKeyString($rgid, $gid, $criteria, $checkPermissions), // @todo stop passing these parameters in & instead calculate them in the merge function based // on the 'real' params like $isRespectExclusions $batchLimit and $isSelected. 'join' => self::getJoinOnDedupeTable(), 'where' => self::getWhereString($batchLimit, $isSelected), ); - return CRM_Dedupe_Merger::merge($dupePairs, $cacheParams, $mode, $autoFlip, $redirectForPerformance); + return CRM_Dedupe_Merger::merge($dupePairs, $cacheParams, $mode, $autoFlip, $redirectForPerformance, $checkPermissions); } /** @@ -679,8 +682,13 @@ CRM_Core_BAO_PrevNextCache::setItem($values); } + /** + * Delete information about merges for the given string. + * + * @param $cacheKeyString + */ public static function resetMergeStats($cacheKeyString) { - return CRM_Core_BAO_PrevNextCache::deleteItem(NULL, "{$cacheKeyString}_stats"); + CRM_Core_BAO_PrevNextCache::deleteItem(NULL, "{$cacheKeyString}_stats"); } public static function getMergeStats($cacheKeyString) { @@ -716,15 +724,18 @@ * A 'safe' value skips the merge if there are any un-resolved conflicts. * Does a force merge otherwise (aggressive mode). * @param bool $autoFlip to let api decide which contact to retain and which to delete. - * Wether to let api decide which contact to retain and which to delete. - * + * Whether to let api decide which contact to retain and which to delete. * * @param bool $redirectForPerformance + * Redirect to a url for batch processing. + * + * @param bool $checkPermissions + * Respect logged in user permissions. * * @return array|bool */ public static function merge($dupePairs = array(), $cacheParams = array(), $mode = 'safe', - $autoFlip = TRUE, $redirectForPerformance = FALSE + $autoFlip = TRUE, $redirectForPerformance = FALSE, $checkPermissions = TRUE ) { $cacheKeyString = CRM_Utils_Array::value('cache_key_string', $cacheParams); $resultStats = array('merged' => array(), 'skipped' => array()); @@ -757,7 +768,7 @@ // Generate var $migrationInfo. The variable structure is exactly same as // $formValues submitted during a UI merge for a pair of contacts. - $rowsElementsAndInfo = CRM_Dedupe_Merger::getRowsElementsAndInfo($mainId, $otherId); + $rowsElementsAndInfo = CRM_Dedupe_Merger::getRowsElementsAndInfo($mainId, $otherId, $checkPermissions); $migrationInfo = &$rowsElementsAndInfo['migration_info']; @@ -769,7 +780,7 @@ // go ahead with merge if there is no conflict $conflicts = array(); if (!CRM_Dedupe_Merger::skipMerge($mainId, $otherId, $migrationInfo, $mode, $conflicts)) { - CRM_Dedupe_Merger::moveAllBelongings($mainId, $otherId, $migrationInfo); + CRM_Dedupe_Merger::moveAllBelongings($mainId, $otherId, $migrationInfo, $checkPermissions); $resultStats['merged'][] = array('main_id' => $mainId, 'other_id' => $otherId); $deletedContacts[] = $otherId; } @@ -1014,6 +1025,14 @@ * Main contact with whom merge has to happen. * @param int $otherId * Duplicate contact which would be deleted after merge operation. + * @param bool $checkPermissions + * Should the logged in user's permissions be ignore. Setting this to false is + * highly risky as it could cause data to be lost due to conflicts not showing up. + * OTOH there is a risk a merger might view custom data they do not have permission to. + * Hence for now only making this really explicit and making it reflect perms in + * an api call. + * + * @todo review permissions issue! * * @return array|bool|int * @@ -1044,8 +1063,11 @@ * to move all fields from the 'other' contact to the 'main' contact, as * though the form had been submitted with those options. * + * @throws \CRM_Core_Exception + * @throws \CiviCRM_API3_Exception + * @throws \Exception */ - public static function getRowsElementsAndInfo($mainId, $otherId) { + public static function getRowsElementsAndInfo($mainId, $otherId, $checkPermissions = TRUE) { $qfZeroBug = 'e8cddb72-a257-11dc-b9cc-0016d3330ee9'; // Fetch contacts @@ -1218,11 +1240,9 @@ foreach ($values['values'] as $index => $value) { $locations[$moniker][$blockName][$cnt] = $value; // Fix address display - $display = ''; if ($blockName == 'address') { CRM_Core_BAO_Address::fixAddress($value); - $display = CRM_Utils_Address::format($value); - $locations[$moniker][$blockName][$cnt]['display'] = $display; + $locations[$moniker][$blockName][$cnt]['display'] = CRM_Utils_Address::format($value); } $cnt++; @@ -1444,11 +1464,11 @@ } // handle custom fields - $mainTree = CRM_Core_BAO_CustomGroup::getTree($main['contact_type'], CRM_Core_DAO::$_nullObject, $mainId, -1, - CRM_Utils_Array::value('contact_sub_type', $main) + $mainTree = CRM_Core_BAO_CustomGroup::getTree($main['contact_type'], NULL, $mainId, -1, + CRM_Utils_Array::value('contact_sub_type', $main), NULL, TRUE, NULL, TRUE, $checkPermissions ); $otherTree = CRM_Core_BAO_CustomGroup::getTree($main['contact_type'], CRM_Core_DAO::$_nullObject, $otherId, -1, - CRM_Utils_Array::value('contact_sub_type', $other) + CRM_Utils_Array::value('contact_sub_type', $other), NULL, TRUE, NULL, TRUE, $checkPermissions ); CRM_Core_DAO::freeResult(); @@ -1515,9 +1535,12 @@ * * @param $migrationInfo * + * @param bool $checkPermissions + * Respect logged in user permissions. + * * @return bool */ - public static function moveAllBelongings($mainId, $otherId, $migrationInfo) { + public static function moveAllBelongings($mainId, $otherId, $migrationInfo, $checkPermissions = TRUE) { if (empty($migrationInfo)) { return FALSE; } @@ -1705,6 +1728,12 @@ $submitted[$key] = CRM_Core_BAO_CustomField::displayValue($value, $fid); break; + case 'Select Date': + if ($cFields[$fid]['attributes']['is_view']) { + $submitted[$key] = date('YmdHis', strtotime($submitted[$key])); + } + break; + case 'CheckBox': case 'AdvMulti-Select': case 'Multi-Select': @@ -1838,22 +1867,16 @@ CRM_Core_BAO_CustomValueTable::setValues($viewOnlyCustomFields); } - if (CRM_Core_Permission::check('merge duplicate contacts') && - CRM_Core_Permission::check('delete contacts') + if (!$checkPermissions || (CRM_Core_Permission::check('merge duplicate contacts') && + CRM_Core_Permission::check('delete contacts')) ) { // if ext id is submitted then set it null for contact to be deleted if (!empty($submitted['external_identifier'])) { $query = "UPDATE civicrm_contact SET external_identifier = null WHERE id = {$otherId}"; CRM_Core_DAO::executeQuery($query); } - civicrm_api3('contact', 'delete', array('id' => $otherId)); - CRM_Core_BAO_PrevNextCache::deleteItem($otherId); } - // FIXME: else part - // else { - // CRM_Core_Session::setStatus( ts('Do not have sufficient permission to delete duplicate contact.') ); - // } // CRM-15681 merge sub_types if ($other_sub_types = CRM_Utils_array::value('contact_sub_type', $migrationInfo['other_details'])) { @@ -2019,24 +2042,27 @@ * @param bool $reloadCacheIfEmpty * @param int $batchLimit * @param bool $isSelected - * @param array $orderByClause + * @param array|string $orderByClause * @param bool $includeConflicts * @param array $criteria * Additional criteria to narrow down the merge group. * + * @param bool $checkPermissions + * Respect logged in user permissions. + * * @return array - * Array of matches meeting the criteria. + * Array of matches meeting the criteria. */ - public static function getDuplicatePairs($rule_group_id, $group_id, $reloadCacheIfEmpty, $batchLimit, $isSelected, $orderByClause = '', $includeConflicts = TRUE, $criteria = array()) { + public static function getDuplicatePairs($rule_group_id, $group_id, $reloadCacheIfEmpty, $batchLimit, $isSelected, $orderByClause = '', $includeConflicts = TRUE, $criteria = array(), $checkPermissions = TRUE) { $where = self::getWhereString($batchLimit, $isSelected); - $cacheKeyString = self::getMergeCacheKeyString($rule_group_id, $group_id, $criteria); + $cacheKeyString = self::getMergeCacheKeyString($rule_group_id, $group_id, $criteria, $checkPermissions); $join = self::getJoinOnDedupeTable(); $dupePairs = CRM_Core_BAO_PrevNextCache::retrieve($cacheKeyString, $join, $where, 0, 0, array(), $orderByClause, $includeConflicts); if (empty($dupePairs) && $reloadCacheIfEmpty) { // If we haven't found any dupes, probably cache is empty. // Try filling cache and give another try. We don't need to specify include conflicts here are there will not be any // until we have done some processing. - CRM_Core_BAO_PrevNextCache::refillCache($rule_group_id, $group_id, $cacheKeyString, $criteria); + CRM_Core_BAO_PrevNextCache::refillCache($rule_group_id, $group_id, $cacheKeyString, $criteria, $checkPermissions); $dupePairs = CRM_Core_BAO_PrevNextCache::retrieve($cacheKeyString, $join, $where, 0, 0, array(), $orderByClause, $includeConflicts); return $dupePairs; } @@ -2052,14 +2078,28 @@ * Additional criteria to narrow down the merge group. * Currently we are only supporting the key 'contact' within it. * + * @param bool $checkPermissions + * Respect the users permissions. + * * @return string */ - public static function getMergeCacheKeyString($rule_group_id, $group_id, $criteria = array()) { + public static function getMergeCacheKeyString($rule_group_id, $group_id, $criteria = array(), $checkPermissions = TRUE) { $contactType = CRM_Dedupe_BAO_RuleGroup::getContactTypeForRuleGroup($rule_group_id); $cacheKeyString = "merge {$contactType}"; $cacheKeyString .= $rule_group_id ? "_{$rule_group_id}" : '_0'; $cacheKeyString .= $group_id ? "_{$group_id}" : '_0'; $cacheKeyString .= !empty($criteria) ? serialize($criteria) : '_0'; + if ($checkPermissions) { + $contactID = CRM_Core_Session::getLoggedInContactID(); + if (!$contactID) { + // Distinguish between no permission check & no logged in user. + $contactID = 'null'; + } + $cacheKeyString .= '_' . $contactID; + } + else { + $cacheKeyString .= '_0'; + } return $cacheKeyString; } diff --git a/CRM/Export/BAO/Export.php b/CRM/Export/BAO/Export.php index 3fd75e4..2c79a8a 100644 --- a/CRM/Export/BAO/Export.php +++ b/CRM/Export/BAO/Export.php @@ -762,9 +762,12 @@ list($outputColumns, $headerRows, $sqlColumns, $metadata) = self::getExportStructureArrays($returnProperties, $query, $phoneTypes, $imProviders, $contactRelationshipTypes, $relationQuery, $selectedPaymentFields); $limitReached = FALSE; + // T120892 I couldn't quite bring myself to put this inside the loop in case an infinite loop + // happened for some reason - which I can't think of - sensible or paranoid? + set_time_limit(ini_get('max_execution_time') + 600); while (!$limitReached) { $limitQuery = "{$queryString} LIMIT {$offset}, {$rowCount}"; - $dao = CRM_Core_DAO::executeQuery($limitQuery); + $dao = CRM_Core_DAO::executeUnbufferedQuery($limitQuery); // If this is less than our limit by the end of the iteration we do not need to run the query again to // check if some remain. $rowsThisIteration = 0; diff --git a/CRM/Logging/Reverter.php b/CRM/Logging/Reverter.php index 5fb317d..cdc184d 100644 --- a/CRM/Logging/Reverter.php +++ b/CRM/Logging/Reverter.php @@ -116,6 +116,7 @@ foreach ($deletes as $table => $ids) { CRM_Core_DAO::executeQuery("DELETE FROM `$table` WHERE id IN (" . implode(', ', array_unique($ids)) . ')'); } + // revert updates by updating to previous values foreach ($reverts as $table => $row) { switch (TRUE) { @@ -132,9 +133,17 @@ if (empty($value) and $value !== 0 and $value !== '0') { $value = 'null'; } + // Date reaches this point in ISO format (possibly) so strip out stuff + // if it does have hyphens of colons demarking the date & it regexes as being a date + // or datetime format. + if (preg_match('/(\d{4})-(\d{2})-(\d{2}) (\d{2}):(\d{2}):(\d{2})/', $value)) { + $value = str_replace('-', '', $value); + $value = str_replace(':', '', $value); + } $dao->$field = $value; } $changes['log_action'] == 'Delete' ? $dao->insert() : $dao->update(); + $dao->reset(); } break; diff --git a/CRM/Report/Form.php b/CRM/Report/Form.php index a018885..e0f813c 100644 --- a/CRM/Report/Form.php +++ b/CRM/Report/Form.php @@ -636,9 +636,6 @@ $this->_instanceButtonName = $this->getButtonName('submit', 'save'); $this->_createNewButtonName = $this->getButtonName('submit', 'next'); - $this->_printButtonName = $this->getButtonName('submit', 'print'); - $this->_pdfButtonName = $this->getButtonName('submit', 'pdf'); - $this->_csvButtonName = $this->getButtonName('submit', 'csv'); $this->_groupButtonName = $this->getButtonName('submit', 'group'); $this->_chartButtonName = $this->getButtonName('submit', 'chart'); } @@ -1468,30 +1465,17 @@ */ public function buildInstanceAndButtons() { CRM_Report_Form_Instance::buildForm($this); - - $label = $this->_id ? ts('Update Report') : ts('Create Report'); - - $this->addElement('submit', $this->_instanceButtonName, $label); - $this->addElement('submit', $this->_printButtonName, ts('Print Report')); - $this->addElement('submit', $this->_pdfButtonName, ts('PDF')); + $this->_actionButtonName = $this->getButtonName('submit'); + $this->addTaskMenu($this->getActions($this->_id)); if ($this->_id) { $this->addElement('submit', $this->_createNewButtonName, ts('Save a Copy') . '...'); } + $this->addElement('submit', $this->_instanceButtonName, + ts('Update Report')); + $this->assign('instanceForm', $this->_instanceForm); - - $label = $this->_id ? ts('Print Report') : ts('Print Preview'); - $this->addElement('submit', $this->_printButtonName, $label); - - $label = $this->_id ? ts('PDF') : ts('Preview PDF'); - $this->addElement('submit', $this->_pdfButtonName, $label); - - $label = $this->_id ? ts('Export to CSV') : ts('Preview CSV'); - - if ($this->_csvSupported) { - $this->addElement('submit', $this->_csvButtonName, $label); - } // CRM-16274 Determine if user has 'edit all contacts' or equivalent $permission = CRM_Core_Permission::getPermission(); @@ -1509,14 +1493,72 @@ $this->addElement('submit', $this->_groupButtonName, '', array('style' => 'display: none;')); $this->addChartOptions(); + $showResultsLabel = $this->getResultsLabel(); $this->addButtons(array( array( 'type' => 'submit', - 'name' => ts('Preview Report'), + 'name' => $showResultsLabel, 'isDefault' => TRUE, ), ) ); + } + + /** + * Has this form been submitted already? + * + * @return bool + */ + public function resultsDisplayed() { + $buttonName = $this->controller->getButtonName(); + return ($buttonName || $this->_outputMode); + } + + /** + * Get the actions for this report instance. + * + * @param int $instanceId + * + * @return array + */ + protected function getActions($instanceId) { + $actions = array( + 'html' => $this->getResultsLabel(), + 'save' => 'Update', + 'copy' => 'Save a Copy', + 'print' => 'Print Report', + 'pdf' => 'Print to PDF', + ); + if (empty($instanceId)) { + $actions['save'] = ts('Create Report'); + } + + if ($this->_outputMode || $this->_id) { + $actions['html'] = 'Refresh Results'; + } + + if ($this->_csvSupported) { + $actions['csv'] = 'Export as CSV'; + } + + if (!empty($this->_charts)) { + $this->assign('charts', $this->_charts); + if ($this->_format != '') { + $actions['tabular'] = 'View as tabular data'; + } + if ($this->_format != 'pieChart') { + $actions['pieChart'] = 'View as pie chart'; + } + if ($this->_format != 'barChart') { + $actions['barChart'] = 'View as bar graph'; + } + } + + if (CRM_Core_Permission::check('administer Reports')) { + $actions['delete'] = 'Delete report'; + } + + return $actions; } /** @@ -2509,6 +2551,7 @@ public function processReportMode() { $this->setOutputMode(); + $buttonName = $this->controller->getButtonName(); $this->_sendmail = CRM_Utils_Request::retrieve( 'sendmail', @@ -2551,6 +2594,7 @@ elseif ($this->_outputMode == 'copy' && $this->_criteriaForm) { $this->_createNew = TRUE; } + $this->assign('outputMode', $this->_outputMode); $this->assign('outputMode', $this->_outputMode); $this->assign('printOnly', $printOnly); @@ -2598,9 +2642,6 @@ if ($this->_outputMode == 'save' || $this->_outputMode == 'copy') { $this->_createNew = ($this->_outputMode == 'copy'); - // Do not pass go. Do not collect another chance to re-run the same query. - // This will be called from the button - there is an earlier response to the url - // perhaps they can still be consolidated more. CRM_Report_Form_Instance::postProcess($this); } $this->beginPostProcessCommon(); @@ -4664,6 +4705,9 @@ LEFT JOIN civicrm_address {$this->_aliases['civicrm_address']} ON ({$this->_aliases['civicrm_contact']}.id = {$this->_aliases['civicrm_address']}.contact_id) AND {$this->_aliases['civicrm_address']}.is_primary = 1\n"; + if ($buttonName == $this->_createNewButtonName) { + $this->_outputMode = 'copy'; + } } } diff --git a/CRM/Report/Form/Contribute/Detail.php b/CRM/Report/Form/Contribute/Detail.php index 60000bf..721c5b2 100644 --- a/CRM/Report/Form/Contribute/Detail.php +++ b/CRM/Report/Form/Contribute/Detail.php @@ -681,7 +681,6 @@ // assign variables to templates $this->doTemplateAssignment($rows); - // do print / pdf / instance stuff if needed $this->endPostProcess($rows); } diff --git a/CRM/Report/Form/Instance.php b/CRM/Report/Form/Instance.php index f0c0846..2d748a3 100644 --- a/CRM/Report/Form/Instance.php +++ b/CRM/Report/Form/Instance.php @@ -358,6 +358,7 @@ 'report_header', 'report_footer', 'grouprole', + 'task', ); foreach ($unsetFields as $field) { unset($formValues[$field]); diff --git a/CRM/Utils/Rule.php b/CRM/Utils/Rule.php index 0ea7c32..c57c0c7 100644 --- a/CRM/Utils/Rule.php +++ b/CRM/Utils/Rule.php @@ -554,7 +554,7 @@ return TRUE; } - return preg_match('/(^-?\d+\.\d?\d?$)|(^-?\.\d\d?$)/', $value) ? TRUE : FALSE; + return preg_match('/(^-?\d+\.?\d*$)|(^-?\d*\.\d+$)/', $value) ? TRUE : FALSE; } /** diff --git a/api/v3/Job.php b/api/v3/Job.php index da4eb04..5ae0363 100644 --- a/api/v3/Job.php +++ b/api/v3/Job.php @@ -490,7 +490,7 @@ $mode = CRM_Utils_Array::value('mode', $params, 'safe'); $autoFlip = CRM_Utils_Array::value('auto_flip', $params, TRUE); - $result = CRM_Dedupe_Merger::batchMerge($rule_group_id, $gid, $mode, $autoFlip, 1, 2, CRM_Utils_Array::value('criteria', $params, array())); + $result = CRM_Dedupe_Merger::batchMerge($rule_group_id, $gid, $mode, $autoFlip, 1, 2, CRM_Utils_Array::value('criteria', $params, array()), CRM_Utils_Array::value('check_permissions', $params)); return civicrm_api3_create_success($result, $params); } diff --git a/packages/DB/common.php b/packages/DB/common.php index e81ead9..18fab27 100644 --- a/packages/DB/common.php +++ b/packages/DB/common.php @@ -1148,6 +1148,28 @@ */ function modifyQuery($query) { + /** + * WMF hack: + * Insert diagnostic info such as requesting user. + * + * Have not found a function to efficiently get uf username. + * Not sure if query begin time is already reported by mysql, or would be useful. + */ + global $installType; + if ( isset( $installType ) ) { + $prefix = "/* Civi utils not available during installation */"; + } + else { + global $user; + if (empty($user)) { + $prefix = 'phpunit-test'; + } + else { + $prefix = "/* https://civicrm.wikimedia.org/user/{$user->uid} */ "; + } + } + $query = $prefix . $query; + return $query; } diff --git a/packages/DB/mysql.php b/packages/DB/mysql.php index f96cd5e..1e837e6 100644 --- a/packages/DB/mysql.php +++ b/packages/DB/mysql.php @@ -829,6 +829,7 @@ 'DELETE FROM \1 WHERE 1=1', $query); } } + $query = parent::modifyQuery($query); return $query; } diff --git a/settings_location.php b/settings_location.php new file mode 100644 index 0000000..e9e9ced --- /dev/null +++ b/settings_location.php @@ -0,0 +1,3 @@ +<?php +define( 'CIVICRM_CONFDIR', realpath( dirname( __FILE__ ) . "/../../../default" )); + diff --git a/templates/CRM/Report/Form.tpl b/templates/CRM/Report/Form.tpl index c5613b0..c580c3c 100644 --- a/templates/CRM/Report/Form.tpl +++ b/templates/CRM/Report/Form.tpl @@ -23,8 +23,6 @@ | see the CiviCRM license FAQ at http://civicrm.org/licensing | +--------------------------------------------------------------------+ *} -{if $outputMode neq 'print'} -{/if} {* this div is being used to apply special css *} {if $section eq 1} <div class="crm-block crm-content-block crm-report-layoutGraph-form-block"> diff --git a/templates/CRM/Report/Form/Actions.tpl b/templates/CRM/Report/Form/Actions.tpl index 1c92130..add64c9 100644 --- a/templates/CRM/Report/Form/Actions.tpl +++ b/templates/CRM/Report/Form/Actions.tpl @@ -27,9 +27,6 @@ {* build the print pdf buttons *} <div class="crm-tasks"> - {assign var=print value="_qf_"|cat:$form.formName|cat:"_submit_print"} - {assign var=pdf value="_qf_"|cat:$form.formName|cat:"_submit_pdf"} - {assign var=csv value="_qf_"|cat:$form.formName|cat:"_submit_csv"} {assign var=group value="_qf_"|cat:$form.formName|cat:"_submit_group"} {assign var=chart value="_qf_"|cat:$form.formName|cat:"_submit_chart"} <table style="border:0;"> @@ -37,9 +34,7 @@ <td> <table class="form-layout-compressed"> <tr> - <td>{$form.$print.html} </td> - <td>{$form.$pdf.html} </td> - <td>{$form.$csv.html} </td> + {include file="CRM/common/tasks.tpl" location="botton"} {if $instanceUrl} <td> » <a href="{$instanceUrl}">{ts}Existing report(s) from this template{/ts}</a></td> {/if} diff --git a/templates/CRM/common/tasks.tpl b/templates/CRM/common/tasks.tpl new file mode 100644 index 0000000..f75dc11 --- /dev/null +++ b/templates/CRM/common/tasks.tpl @@ -0,0 +1,4 @@ +<span id='task-section'> + {$form.task.html} + {$form.$actionButtonName.html} +</span> \ No newline at end of file -- To view, visit https://gerrit.wikimedia.org/r/293891 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I37c9a5af150e0d4ae91841f50bc24e488fc729c3 Gerrit-PatchSet: 1 Gerrit-Project: wikimedia/fundraising/crm/civicrm Gerrit-Branch: master Gerrit-Owner: Eileen <emcnaugh...@wikimedia.org> _______________________________________________ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
