jenkins-bot has submitted this change and it was merged.

Change subject: API BREAKING CHANGE - Remove rawMode use
......................................................................


API BREAKING CHANGE - Remove rawMode use

This is a breaking api change! :D

This also removes ALL BUT ONE usages of the
deprecated rawMode stuff from the mediawiki API

The final use will be removed in a follouwp

Breaks can be seen below:

 - XML output aliases are now grouped by language
 - XML output may no longer give elements when they
   are empty
 - XML any claim, qualifer, reference or snak elements
   that had an '_idx' element will no longer have it
 - ALL output may now give empty elements, ie. labels
   when an entity has none

Once merged this will be announced.

DEPLOY: This is an API breaking change...

This will likely also cause a slight increase in the
size of the JSON log files due to the fact that
empty elements in the serliazation will no longer
be removed.
After compression the difference should be negligible

Bug: T95168
Change-Id: I6b8a041917304bb9c78153da5bd71327b83699c6
---
M lib/includes/serialization/CallbackFactory.php
M repo/includes/LinkedData/EntityDataSerializationService.php
M repo/includes/api/ApiHelperFactory.php
M repo/includes/api/GetEntities.php
M repo/includes/api/ResultBuilder.php
M repo/tests/phpunit/data/api/editentity.xml
M repo/tests/phpunit/data/api/getclaims.xml
M repo/tests/phpunit/data/api/getentities.xml
M repo/tests/phpunit/data/api/setaliases-removed.xml
M repo/tests/phpunit/data/api/setaliases.xml
M repo/tests/phpunit/data/api/setclaim.xml
M repo/tests/phpunit/data/api/setqualifier.xml
M repo/tests/phpunit/data/api/setreference.xml
M repo/tests/phpunit/includes/api/ApiHelperFactoryTest.php
M repo/tests/phpunit/includes/api/ApiXmlFormatTest.php
M repo/tests/phpunit/includes/api/ModifyTermTestCase.php
M repo/tests/phpunit/includes/api/ResultBuilderTest.php
M repo/tests/phpunit/includes/api/SetAliasesTest.php
M repo/tests/phpunit/includes/api/SetSiteLinkTest.php
19 files changed, 617 insertions(+), 870 deletions(-)

Approvals:
  Aude: Looks good to me, approved
  Daniel Kinzler: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/lib/includes/serialization/CallbackFactory.php 
b/lib/includes/serialization/CallbackFactory.php
index 10c591a..4dc81f2 100644
--- a/lib/includes/serialization/CallbackFactory.php
+++ b/lib/includes/serialization/CallbackFactory.php
@@ -30,6 +30,23 @@
        }
 
        /**
+        * Get callable to index array with the given tag name
+        *
+        * @param string $type
+        * @param string $kvpKeyName
+        *
+        * @return callable
+        */
+       public function getCallbackToSetArrayType( $type, $kvpKeyName = null ) {
+               return function( $array ) use ( $type, $kvpKeyName ) {
+                       if ( is_array( $array ) ) {
+                               ApiResult::setArrayType( $array, $type, 
$kvpKeyName );
+                       }
+                       return $array;
+               };
+       }
+
+       /**
         * Get callable to remove array keys and optionally set the key as an 
array value
         *
         * @param string|null $addAsArrayElement
@@ -68,11 +85,13 @@
 
        public function getCallbackToAddDataTypeToSnak( PropertyDataTypeLookup 
$dataTypeLookup ) {
                return function ( $array ) use ( $dataTypeLookup ) {
-                       try {
-                               $dataType = 
$dataTypeLookup->getDataTypeIdForProperty( new PropertyId( $array['property'] ) 
);
-                               $array['datatype'] = $dataType;
-                       } catch ( PropertyNotFoundException $e ) {
-                               //XXX: shall we set $serialization['datatype'] 
= 'bad' ??
+                       if ( is_array( $array ) ) {
+                               try {
+                                       $dataType = 
$dataTypeLookup->getDataTypeIdForProperty( new PropertyId( $array['property'] ) 
);
+                                       $array['datatype'] = $dataType;
+                               } catch ( PropertyNotFoundException $e ) {
+                                       //XXX: shall we set 
$serialization['datatype'] = 'bad' ??
+                               }
                        }
                        return $array;
                };
diff --git a/repo/includes/LinkedData/EntityDataSerializationService.php 
b/repo/includes/LinkedData/EntityDataSerializationService.php
index 86db204..01b11d3 100644
--- a/repo/includes/LinkedData/EntityDataSerializationService.php
+++ b/repo/includes/LinkedData/EntityDataSerializationService.php
@@ -459,7 +459,7 @@
                        $this->serializerFactory,
                        $this->siteStore,
                        $this->propertyLookup,
-                       false // Never index tags for this service as we dont 
output XML
+                       false // Never add meta data for this service
                );
                $resultBuilder->addEntityRevision( null, $entityRevision );
 
diff --git a/repo/includes/api/ApiHelperFactory.php 
b/repo/includes/api/ApiHelperFactory.php
index 4d4ba68..7dc0557 100644
--- a/repo/includes/api/ApiHelperFactory.php
+++ b/repo/includes/api/ApiHelperFactory.php
@@ -98,7 +98,8 @@
                        $this->titleLookup,
                        $this->newSerializerFactory(),
                        $this->siteStore,
-                       $this->dataTypeLookup
+                       $this->dataTypeLookup,
+                       true // The mediawiki api should always be given 
metadata
                );
        }
 
diff --git a/repo/includes/api/GetEntities.php 
b/repo/includes/api/GetEntities.php
index 6c7b244..39cfef1 100644
--- a/repo/includes/api/GetEntities.php
+++ b/repo/includes/api/GetEntities.php
@@ -128,9 +128,6 @@
                        $this->handleEntity( $sourceEntityId, $entityRevision, 
$params );
                }
 
-               //todo remove once result builder is used... (what exactly does 
this do....?)
-               $this->getResult()->addIndexedTagName( array( 'entities' ), 
'entity' );
-
                $this->resultBuilder->markSuccess( 1 );
        }
 
diff --git a/repo/includes/api/ResultBuilder.php 
b/repo/includes/api/ResultBuilder.php
index d08c5c6..3012e47 100644
--- a/repo/includes/api/ResultBuilder.php
+++ b/repo/includes/api/ResultBuilder.php
@@ -61,9 +61,8 @@
 
        /**
         * @var bool|null when special elements such as '_element' are needed 
by the formatter.
-        * @note please use $this->getIsRawMode() to access this value!
         */
-       private $isRawMode;
+       private $addMetaData;
 
        /**
         * @var SerializationModifier
@@ -86,7 +85,7 @@
         * @param SerializerFactory $serializerFactory
         * @param SiteStore $siteStore
         * @param PropertyDataTypeLookup $dataTypeLookup
-        * @param bool $isRawMode when special elements such as '_element' are 
needed by the formatter.
+        * @param bool $addMetaData when special elements such as '_element' 
are needed
         */
        public function __construct(
                ApiResult $result,
@@ -94,32 +93,16 @@
                SerializerFactory $serializerFactory,
                SiteStore $siteStore,
                PropertyDataTypeLookup $dataTypeLookup,
-               $isRawMode = null
+               $addMetaData = null
        ) {
                $this->result = $result;
                $this->entityTitleLookup = $entityTitleLookup;
                $this->serializerFactory = $serializerFactory;
                $this->siteStore = $siteStore;
                $this->dataTypeLookup = $dataTypeLookup;
-               $this->isRawMode = $isRawMode;
+               $this->addMetaData = $addMetaData;
                $this->modifier = new SerializationModifier();
                $this->callbackFactory = new CallbackFactory();
-       }
-
-       /**
-        * isRawMode needs to be lazy initialized since ApiMain may set it
-        * on the ApiResult after the module is constructed.
-        *
-        * isRawMode can be set in the constructor, such as for testing 
purposes.
-        *
-        * @return bool
-        */
-       private function getIsRawMode() {
-               if ( $this->isRawMode === null ) {
-                       $this->isRawMode = $this->result->getIsRawMode();
-               }
-
-               return $this->isRawMode;
        }
 
        /**
@@ -155,20 +138,20 @@
         * @param $path array|string|null
         * @param $name string
         * @param $values array
-        * @param string $tag tag name to use for elements of $values
+        * @param string $tag tag name to use for elements of $values if not 
already present
         */
        public function setList( $path, $name, array $values, $tag ) {
                $this->checkPathType( $path );
                Assert::parameterType( 'string', $name, '$name' );
                Assert::parameterType( 'string', $tag, '$tag' );
 
-               if ( $this->getIsRawMode() ) {
-                       // Unset first, so we don't make the tag name an actual 
value.
-                       // We'll be setting this to $tag by calling 
setIndexedTagName().
-                       unset( $values['_element'] );
-
-                       $values = array_values( $values );
-                       ApiResult::setIndexedTagName( $values, $tag );
+               if ( $this->addMetaData ) {
+                       if ( !array_key_exists( ApiResult::META_TYPE, $values ) 
) {
+                               ApiResult::setArrayType( $values, 'array' );
+                       }
+                       if ( !array_key_exists( 
ApiResult::META_INDEXED_TAG_NAME, $values ) ) {
+                               ApiResult::setIndexedTagName( $values, $tag );
+                       }
                }
 
                $this->result->addValue( $path, $name, $values );
@@ -213,7 +196,7 @@
         *
         * @param $path array|string|null
         * @param $key int|string|null the key to use when appending, or null 
for automatic.
-        * May be ignored even if given, based on $this->result->getIsRawMode().
+        * May be ignored even if given, based on $this->addMetaData.
         * @param $value mixed
         * @param string $tag tag name to use for $value in indexed mode
         */
@@ -223,12 +206,10 @@
                Assert::parameterType( 'string', $tag, '$tag' );
                $this->checkValueIsNotList( $value );
 
-               if ( $this->getIsRawMode() ) {
-                       $key = null;
-               }
-
                $this->result->addValue( $path, $key, $value );
-               $this->result->addIndexedTagName( $path, $tag );
+               if ( $this->addMetaData ) {
+                       $this->result->addIndexedTagName( $path, $tag );
+               }
        }
 
        /**
@@ -328,6 +309,9 @@
                }
 
                $this->appendValue( array( 'entities' ), 
$sourceEntityIdSerialization, $record, 'entity' );
+               if ( $this->addMetaData ) {
+                       $this->result->addArrayType( array( 'entities' ), 
'array' );
+               }
        }
 
        /**
@@ -367,32 +351,10 @@
                        $filterLangCodes
                );
 
-               if ( $this->getIsRawMode() ) {
-                       $serialization = $this->getRawModeEntitySerialization( 
$serialization );
-               } else {
-                       // Non raw mode formats dont want empty parts....
-                       $serialization = 
$this->filterEmptyEntitySerializationParts( $serialization );
+               if ( $this->addMetaData ) {
+                       $serialization = 
$this->getEntitySerializationWithMetaData( $serialization );
                }
 
-               return $serialization;
-       }
-
-       private function filterEmptyEntitySerializationParts( array 
$serialization ) {
-               if ( empty( $serialization['labels'] ) ) {
-                       unset( $serialization['labels'] );
-               }
-               if ( empty( $serialization['descriptions'] ) ) {
-                       unset( $serialization['descriptions'] );
-               }
-               if ( empty( $serialization['aliases'] ) ) {
-                       unset( $serialization['aliases'] );
-               }
-               if ( empty( $serialization['claims'] ) ) {
-                       unset( $serialization['claims'] );
-               }
-               if ( empty( $serialization['sitelinks'] ) ) {
-                       unset( $serialization['sitelinks'] );
-               }
                return $serialization;
        }
 
@@ -548,55 +510,31 @@
                return $serialization;
        }
 
-       private function getRawModeEntitySerialization( $serialization ) {
-               // In raw mode aliases are not currently grouped by language
-               $serialization = $this->modifier->modifyUsingCallback(
-                       $serialization,
-                       'aliases',
-                       function( $array ) {
-                               $newArray = array();
-                               foreach ( $array as $aliasGroup ) {
-                                       foreach ( $aliasGroup as $alias ) {
-                                               $newArray[] = $alias;
-                                       }
-                               }
-                               return $newArray;
-                       }
-               );
-               // In the old Lib serializers
-               $serialization = $this->modifier->modifyUsingCallback(
-                       $serialization,
-                       'claims/*/*',
-                       function( $array ) {
-                               if ( !array_key_exists( 'qualifiers', $array ) 
) {
-                                       $array['qualifiers'] = array();
-                               }
-                               if ( !array_key_exists( 'qualifiers-order', 
$array ) ) {
-                                       $array['qualifiers-order'] = array();
-                               }
-                               return $array;
-                       }
-               );
-               $keysToValues = array(
-                       'aliases' => null,
-                       'descriptions' => null,
-                       'labels' => null,
+       private function getEntitySerializationWithMetaData( $serialization ) {
+               $arrayTypes = array(
+                       'aliases' => 'id',
                        'claims/*/*/references/*/snaks' => 'id',
                        'claims/*/*/qualifiers' => 'id',
                        'claims' => 'id',
+                       'descriptions' => null,
+                       'labels' => null,
                        'sitelinks' => null,
                );
-               foreach ( $keysToValues as $path => $newKey ) {
+               foreach ( $arrayTypes as $path => $keyName ) {
                        $serialization = $this->modifier->modifyUsingCallback(
                                $serialization,
                                $path,
-                               
$this->callbackFactory->getCallbackToRemoveKeys( $newKey )
+                               
$this->callbackFactory->getCallbackToSetArrayType(
+                                       ( $keyName === null ? 'array' : 'kvp' ),
+                                       $keyName
+                               )
                        );
                }
-               $tagsToAdd = array(
+               $indexTags = array(
                        'labels' => 'label',
                        'descriptions' => 'description',
-                       'aliases' => 'alias',
+                       'aliases/*' => 'alias',
+                       'aliases' => 'language',
                        'sitelinks/*/badges' => 'badge',
                        'sitelinks' => 'sitelink',
                        'claims/*/*/qualifiers/*' => 'qualifiers',
@@ -609,7 +547,7 @@
                        'claims/*' => 'claim',
                        'claims' => 'property',
                );
-               foreach ( $tagsToAdd as $path => $tag ) {
+               foreach ( $indexTags as $path => $tag ) {
                        $serialization = $this->modifier->modifyUsingCallback(
                                $serialization,
                                $path,
@@ -719,17 +657,24 @@
         * @param array|string $path where the data is located
         */
        public function addAliasGroupList( AliasGroupList $aliasGroupList, 
$path ) {
-               if ( $this->getIsRawMode() ) {
-                       $serializer = 
$this->serializerFactory->newAliasGroupSerializer();
-                       $values = array();
-                       foreach ( $aliasGroupList->toArray() as $aliasGroup ) {
-                               $values = array_merge( $values, 
$serializer->serialize( $aliasGroup ) );
-                       }
-               } else {
-                       $serializer = 
$this->serializerFactory->newAliasGroupListSerializer();
-                       $values = $serializer->serialize( $aliasGroupList );
+               $serializer = 
$this->serializerFactory->newAliasGroupListSerializer();
+               $values = $serializer->serialize( $aliasGroupList );
+
+               if ( $this->addMetaData ) {
+                       $values = $this->modifier->modifyUsingCallback(
+                               $values,
+                               null,
+                               
$this->callbackFactory->getCallbackToSetArrayType( 'kvp', 'id' )
+                       );
+                       $values = $this->modifier->modifyUsingCallback(
+                               $values,
+                               '*',
+                               $this->callbackFactory->getCallbackToIndexTags( 
'alias' )
+                       );
                }
-               $this->setList( $path, 'aliases', $values, 'alias' );
+
+               $this->setList( $path, 'aliases', $values, 'language' );
+               ApiResult::setArrayType( $values, 'kvp', 'id' );
        }
 
        /**
@@ -755,8 +700,8 @@
                        $values = $this->getSiteLinkListArrayWithUrls( $values 
);
                }
 
-               if ( $this->getIsRawMode() ) {
-                       $values = $this->getRawModeSiteLinkListArray( $values );
+               if ( $this->addMetaData ) {
+                       $values = $this->getSiteLinkListArrayWithMetaData( 
$values );
                }
 
                $this->setList( $path, 'sitelinks', $values, 'sitelink' );
@@ -774,13 +719,18 @@
                return $this->modifier->modifyUsingCallback( $array, '*', 
$addUrlCallback );
        }
 
-       private function getRawModeSiteLinkListArray( array $array ) {
-               $addIndexedBadgesCallback = function ( $array ) {
-                       ApiResult::setIndexedTagName( $array, 'badge' );
-                       return $array;
-               };
-               $array = array_values( $array );
-               return $this->modifier->modifyUsingCallback( $array, 
'*/badges', $addIndexedBadgesCallback );
+       private function getSiteLinkListArrayWithMetaData( array $array ) {
+               $array = $this->modifier->modifyUsingCallback(
+                       $array,
+                       null,
+                       $this->callbackFactory->getCallbackToSetArrayType( 
'array' )
+               );
+               $array = $this->modifier->modifyUsingCallback(
+                       $array,
+                       '*/badges',
+                       $this->callbackFactory->getCallbackToIndexTags( 'badge' 
)
+               );
+               return $array;
        }
 
        /**
@@ -798,6 +748,13 @@
                        $value = $serializer->serialize( $siteLink );
                        $value['removed'] = '';
                        $values[$siteLink->getSiteId()] = $value;
+               }
+               if ( $this->addMetaData ) {
+                       $values = $this->modifier->modifyUsingCallback(
+                               $values,
+                               null,
+                               
$this->callbackFactory->getCallbackToSetArrayType( 'array' )
+                       );
                }
                $this->setList( $path, 'sitelinks', $values, 'sitelink' );
        }
@@ -827,21 +784,30 @@
                        );
                }
 
-               if ( !$this->getIsRawMode() ) {
-                       $values = $this->getArrayWithAlteredClaims( $values, 
false, '*/*/' );
-               } else {
-                       $values = $this->getArrayWithAlteredClaims( $values, 
true, '*/*/' );
-                       $values = $this->getArrayWithRawModeClaims( $values, 
'*/*/' );
+               $values = $this->getArrayWithAlteredClaims( $values, '*/*/' );
+
+               if ( $this->addMetaData ) {
+                       $values = $this->getClaimsArrayWithMetaData( $values, 
'*/*/' );
                        $values = $this->modifier->modifyUsingCallback(
                                $values,
                                null,
-                               
$this->callbackFactory->getCallbackToRemoveKeys( 'id' )
+                               
$this->callbackFactory->getCallbackToSetArrayType( 'kvp', 'id' )
                        );
                        $values = $this->modifier->modifyUsingCallback(
                                $values,
                                '*',
                                $this->callbackFactory->getCallbackToIndexTags( 
'claim' )
                        );
+               }
+
+               $values = $this->modifier->modifyUsingCallback(
+                       $values,
+                       '*/*/mainsnak',
+                       $this->callbackFactory->getCallbackToAddDataTypeToSnak( 
$this->dataTypeLookup )
+               );
+
+               if ( $this->addMetaData ) {
+                       ApiResult::setArrayType( $values, 'kvp', 'id' );
                }
 
                $this->setList( $path, 'claims', $values, 'property' );
@@ -865,48 +831,29 @@
 
                $value = $this->getArrayWithAlteredClaims( $value );
 
-               if ( $this->getIsRawMode() ) {
-                       $value = $this->getArrayWithRawModeClaims( $value );
+               if ( $this->addMetaData ) {
+                       $value = $this->getClaimsArrayWithMetaData( $value );
                }
+
+               $value = $this->modifier->modifyUsingCallback(
+                       $value,
+                       'mainsnak',
+                       $this->callbackFactory->getCallbackToAddDataTypeToSnak( 
$this->dataTypeLookup )
+               );
 
                $this->setValue( null, 'claim', $value );
        }
 
        /**
         * @param array $array
-        * @param bool $allowEmptyQualifiers
         * @param string $claimPath to the claim array/arrays with trailing /
         *
         * @return array
         */
        private function getArrayWithAlteredClaims(
                array $array,
-               $allowEmptyQualifiers = true,
                $claimPath = ''
        ) {
-               if ( $allowEmptyQualifiers ) {
-                       /**
-                        * Below we force an empty qualifiers and 
qualifiers-order element in the output.
-                        * This is to make sure we dont break anything that 
assumes this is always here.
-                        * This hack was added when moving away from the Lib 
serializers
-                        * TODO: remove this hack when we make other 'breaking 
changes' to the api output
-                        */
-                       $array = $this->modifier->modifyUsingCallback(
-                               $array,
-                               trim( $claimPath, '/' ),
-                               function ( $array ) {
-                                       if ( !isset( $array['qualifiers'] ) ) {
-                                               $array['qualifiers'] = array();
-                                       }
-                                       if ( !isset( $array['qualifiers-order'] 
) ) {
-                                               $array['qualifiers-order'] = 
array();
-                                       }
-
-                                       return $array;
-                               }
-                       );
-               }
-
                $array = $this->getArrayWithDataTypesInGroupedSnakListAtPath(
                        $array,
                        $claimPath . 'references/*/snaks'
@@ -929,13 +876,13 @@
         *
         * @return array
         */
-       private function getArrayWithRawModeClaims( array $array, $claimPath = 
'' ) {
-               $rawModeModifications = array(
+       private function getClaimsArrayWithMetaData( array $array, $claimPath = 
'' ) {
+               $metaDataModifications = array(
                        'references/*/snaks/*' => array(
                                $this->callbackFactory->getCallbackToIndexTags( 
'snak' ),
                        ),
                        'references/*/snaks' => array(
-                               
$this->callbackFactory->getCallbackToRemoveKeys( 'id' ),
+                               
$this->callbackFactory->getCallbackToSetArrayType( 'kvp', 'id' ),
                                $this->callbackFactory->getCallbackToIndexTags( 
'property' ),
                        ),
                        'references/*/snaks-order' => array(
@@ -948,7 +895,7 @@
                                $this->callbackFactory->getCallbackToIndexTags( 
'qualifiers' ),
                        ),
                        'qualifiers' => array(
-                               
$this->callbackFactory->getCallbackToRemoveKeys( 'id' ),
+                               
$this->callbackFactory->getCallbackToSetArrayType( 'kvp', 'id' ),
                                $this->callbackFactory->getCallbackToIndexTags( 
'property' ),
                        ),
                        'qualifiers-order' => array(
@@ -959,7 +906,7 @@
                        ),
                );
 
-               foreach ( $rawModeModifications as $path => $callbacks ) {
+               foreach ( $metaDataModifications as $path => $callbacks ) {
                        foreach ( $callbacks as $callback ) {
                                $array = $this->modifier->modifyUsingCallback( 
$array, $claimPath . $path, $callback );
                        }
@@ -986,8 +933,8 @@
 
                $value = $this->getArrayWithDataTypesInGroupedSnakListAtPath( 
$value, 'snaks' );
 
-               if ( $this->getIsRawMode() ) {
-                       $value = $this->getRawModeReferenceArray( $value );
+               if ( $this->addMetaData ) {
+                       $value = $this->getReferenceArrayWithMetaData( $value );
                }
 
                $this->setValue( null, 'reference', $value );
@@ -1007,17 +954,19 @@
                );
        }
 
-       private function getRawModeReferenceArray( $array ) {
+       private function getReferenceArrayWithMetaData( $array ) {
                $array = $this->modifier->modifyUsingCallback( $array, 
'snaks-order', function ( $array ) {
                        ApiResult::setIndexedTagName( $array, 'property' );
                        return $array;
                } );
                $array = $this->modifier->modifyUsingCallback( $array, 'snaks', 
function ( $array ) {
                        foreach ( $array as $propertyIdGroup => &$snakGroup ) {
-                               $snakGroup['id'] = $propertyIdGroup;
-                               ApiResult::setIndexedTagName( $snakGroup, 
'snak' );
+                               if ( is_array( $snakGroup ) ) {
+                                       ApiResult::setArrayType( $array, 
'array' );
+                                       ApiResult::setIndexedTagName( 
$snakGroup, 'snak' );
+                               }
                        }
-                       $array = array_values( $array );
+                       ApiResult::setArrayType( $array, 'kvp', 'id' );
                        ApiResult::setIndexedTagName( $array, 'property' );
                        return $array;
                } );
@@ -1050,6 +999,10 @@
                        'entity'
                );
 
+               if ( $this->addMetaData ) {
+                       $this->result->addIndexedTagName( 'entities', 'entity' 
);
+               }
+
                $this->missingEntityCounter--;
        }
 
diff --git a/repo/tests/phpunit/data/api/editentity.xml 
b/repo/tests/phpunit/data/api/editentity.xml
index 6862044..7b2983b 100644
--- a/repo/tests/phpunit/data/api/editentity.xml
+++ b/repo/tests/phpunit/data/api/editentity.xml
@@ -8,21 +8,23 @@
                        <description language="fr" value="constructeur 
a&#xE9;ronautique europ&#xE9;en"/>
                </descriptions>
                <aliases>
-                       <alias language="uk" value="Airbus S.A.S"/>
-                       <alias language="uk" 
value="&#x410;&#x435;&#x440;&#x43E;&#x301;&#x431;&#x443;&#x441;"/>
+                       <language  id="uk">
+                               <alias language="uk" value="Airbus S.A.S"/>
+                               <alias language="uk" 
value="&#x410;&#x435;&#x440;&#x43E;&#x301;&#x431;&#x443;&#x441;"/>
+                       </language>
                </aliases>
                <sitelinks/>
                <claims>
                        <property id="$propertyIdUnderTest">
-                               <claim _idx="0" 
id="$itemIdUnderTest$26D08148-75F5-4AD1-B340-3A6A2F32C202" type="statement"
+                               <claim type="statement" 
id="$itemIdUnderTest$26D08148-75F5-4AD1-B340-3A6A2F32C202"
                                           rank="normal">
                                        <mainsnak snaktype="value" 
property="$propertyIdUnderTest" datatype="string">
                                                <datavalue value="SomeMainVal" 
type="string"/>
                                        </mainsnak>
                                        <qualifiers>
                                                <property 
id="$propertyIdUnderTest">
-                                                       <qualifiers _idx="0" 
hash="XXX"
-                                                                               
snaktype="value" property="$propertyIdUnderTest" datatype="string">
+                                                       <qualifiers 
snaktype="value" property="$propertyIdUnderTest"
+                                                                               
hash="XXX" datatype="string">
                                                                <datavalue 
value="SomeQualVal" type="string"/>
                                                        </qualifiers>
                                                </property>
@@ -34,7 +36,7 @@
                                                <reference hash="XXX">
                                                        <snaks>
                                                                <property 
id="$propertyIdUnderTest">
-                                                                       <snak 
_idx="0" snaktype="value" property="$propertyIdUnderTest"
+                                                                       <snak 
snaktype="value" property="$propertyIdUnderTest"
                                                                                
  datatype="string">
                                                                                
<datavalue value="SomeRefVal" type="string"/>
                                                                        </snak>
diff --git a/repo/tests/phpunit/data/api/getclaims.xml 
b/repo/tests/phpunit/data/api/getclaims.xml
index dbaef59..379735d 100644
--- a/repo/tests/phpunit/data/api/getclaims.xml
+++ b/repo/tests/phpunit/data/api/getclaims.xml
@@ -2,10 +2,8 @@
 <api>
        <claims>
                <property id="$propertyIdUnderTest">
-                       <claim _idx="0" type="statement" 
id="$itemIdUnderTest$1111AAAA-43cb-ed6d-3adb-760e85bd17ee" rank="normal">
+                       <claim type="statement" 
id="$itemIdUnderTest$1111AAAA-43cb-ed6d-3adb-760e85bd17ee" rank="normal">
                                <mainsnak snaktype="novalue" 
property="$propertyIdUnderTest" datatype="string"/>
-                               <qualifiers/>
-                               <qualifiers-order/>
                        </claim>
                </property>
        </claims>
diff --git a/repo/tests/phpunit/data/api/getentities.xml 
b/repo/tests/phpunit/data/api/getentities.xml
index 4ecdf20..71b4dbf 100644
--- a/repo/tests/phpunit/data/api/getentities.xml
+++ b/repo/tests/phpunit/data/api/getentities.xml
@@ -3,25 +3,27 @@
        <entities>
                <entity type="item" id="$itemIdUnderTest">
                        <labels>
-                               <label language="en" value="en-label"/>
                                <label language="de" value="de-label"/>
+                               <label language="en" value="en-label"/>
                        </labels>
                        <descriptions>
                                <description language="de" value="de-desc"/>
                                <description language="es" value="es-desc"/>
                        </descriptions>
                        <aliases>
-                               <alias language="pt" value="AA"/>
-                               <alias language="pt" value="BB"/>
-                               <alias language="en" value="AA-en"/>
-                               <alias language="en" value="BB-en"/>
+                               <language id="pt">
+                                       <alias language="pt" value="AA"/>
+                                       <alias language="pt" value="BB"/>
+                               </language>
+                               <language id="en">
+                                       <alias language="en" value="AA-en"/>
+                                       <alias language="en" value="BB-en"/>
+                       </language>
                        </aliases>
                        <claims>
                                <property id="$propertyIdUnderTest">
-                                       <claim _idx="0" type="statement" 
id="$itemIdUnderTest$1111AAAA-43cb-ed6d-3adb-760e85bd17ee" rank="normal">
+                                       <claim type="statement" 
id="$itemIdUnderTest$1111AAAA-43cb-ed6d-3adb-760e85bd17ee" rank="normal">
                                                <mainsnak snaktype="novalue" 
property="$propertyIdUnderTest" datatype="string"/>
-                                               <qualifiers/>
-                                               <qualifiers-order/>
                                        </claim>
                                </property>
                        </claims>
diff --git a/repo/tests/phpunit/data/api/setaliases-removed.xml 
b/repo/tests/phpunit/data/api/setaliases-removed.xml
index c13b853..f7c32e5 100644
--- a/repo/tests/phpunit/data/api/setaliases-removed.xml
+++ b/repo/tests/phpunit/data/api/setaliases-removed.xml
@@ -2,7 +2,9 @@
 <api success="1">
        <entity id="$itemIdUnderTest" type="item">
                <aliases>
-                       <alias language="en-gb" value="AA"/>
+                       <language id="en-gb">
+                               <alias language="en-gb" value="AA"/>
+                       </language>
                </aliases>
        </entity>
 </api>
\ No newline at end of file
diff --git a/repo/tests/phpunit/data/api/setaliases.xml 
b/repo/tests/phpunit/data/api/setaliases.xml
index 301fe19..20e2c9f 100644
--- a/repo/tests/phpunit/data/api/setaliases.xml
+++ b/repo/tests/phpunit/data/api/setaliases.xml
@@ -2,9 +2,11 @@
 <api success="1">
        <entity id="$itemIdUnderTest" type="item">
                <aliases>
-                       <alias language="en-gb" value="AA"/>
-                       <alias language="en-gb" value="BB"/>
-                       <alias language="en-gb" value="CC"/>
+                       <language id="en-gb" >
+                               <alias language="en-gb" value="AA"/>
+                               <alias language="en-gb" value="BB"/>
+                               <alias language="en-gb" value="CC"/>
+                       </language>
                </aliases>
        </entity>
 </api>
\ No newline at end of file
diff --git a/repo/tests/phpunit/data/api/setclaim.xml 
b/repo/tests/phpunit/data/api/setclaim.xml
index 22e937b..7b62ca3 100644
--- a/repo/tests/phpunit/data/api/setclaim.xml
+++ b/repo/tests/phpunit/data/api/setclaim.xml
@@ -5,7 +5,5 @@
                <mainsnak snaktype="value" property="$propertyIdUnderTest" 
datatype="string">
                        <datavalue value="SetClaimValue" type="string"/>
                </mainsnak>
-               <qualifiers/>
-               <qualifiers-order/>
        </claim>
 </api>
\ No newline at end of file
diff --git a/repo/tests/phpunit/data/api/setqualifier.xml 
b/repo/tests/phpunit/data/api/setqualifier.xml
index 23277af..a492337 100644
--- a/repo/tests/phpunit/data/api/setqualifier.xml
+++ b/repo/tests/phpunit/data/api/setqualifier.xml
@@ -1,12 +1,11 @@
 <?xml version="1.0"?>
 <api success="1">
        <pageinfo/>
-       <claim id="$itemIdUnderTest$1111AAAA-43cb-ed6d-3adb-760e85bd17ee" 
type="statement" rank="normal">
+       <claim type="statement" 
id="$itemIdUnderTest$1111AAAA-43cb-ed6d-3adb-760e85bd17ee" rank="normal">
                <mainsnak snaktype="novalue" property="$propertyIdUnderTest" 
datatype="string"/>
                <qualifiers>
                        <property id="$propertyIdUnderTest">
-                               <qualifiers _idx="0" hash="XXX"
-                                                       snaktype="value" 
property="$propertyIdUnderTest" datatype="string">
+                               <qualifiers snaktype="value" 
property="$propertyIdUnderTest" hash="XXX" datatype="string">
                                        <datavalue value="QualiValue" 
type="string"/>
                                </qualifiers>
                        </property>
diff --git a/repo/tests/phpunit/data/api/setreference.xml 
b/repo/tests/phpunit/data/api/setreference.xml
index ed518b0..13410be 100644
--- a/repo/tests/phpunit/data/api/setreference.xml
+++ b/repo/tests/phpunit/data/api/setreference.xml
@@ -4,10 +4,10 @@
        <reference hash="XXX">
                <snaks>
                        <property id="$propertyIdUnderTest">
-                               <snak _idx="0" snaktype="value" 
property="$propertyIdUnderTest" datatype="string">
+                               <snak snaktype="value" 
property="$propertyIdUnderTest" datatype="string">
                                        <datavalue value="Val1" type="string"/>
                                </snak>
-                               <snak _idx="1" snaktype="value" 
property="$propertyIdUnderTest" datatype="string">
+                               <snak snaktype="value" 
property="$propertyIdUnderTest" datatype="string">
                                        <datavalue value="Val2" type="string"/>
                                </snak>
                        </property>
diff --git a/repo/tests/phpunit/includes/api/ApiHelperFactoryTest.php 
b/repo/tests/phpunit/includes/api/ApiHelperFactoryTest.php
index 13e5e0c..2e09090 100644
--- a/repo/tests/phpunit/includes/api/ApiHelperFactoryTest.php
+++ b/repo/tests/phpunit/includes/api/ApiHelperFactoryTest.php
@@ -49,10 +49,6 @@
                        ->disableOriginalConstructor()
                        ->getMock();
 
-               $result->expects( $this->any() )
-                       ->method( 'getIsRawMode' )
-                       ->will( $this->returnValue( false ) );
-
                $api = $this->getMockBuilder( 'ApiBase' )
                        ->disableOriginalConstructor()
                        ->getMock();
diff --git a/repo/tests/phpunit/includes/api/ApiXmlFormatTest.php 
b/repo/tests/phpunit/includes/api/ApiXmlFormatTest.php
index c325063..52838a3 100644
--- a/repo/tests/phpunit/includes/api/ApiXmlFormatTest.php
+++ b/repo/tests/phpunit/includes/api/ApiXmlFormatTest.php
@@ -351,8 +351,7 @@
 
        /**
         * This mimics ApiMain::executeAction with the relevant parts,
-        * including setupExternalResponse where the printer is set. and
-        * Then raw mode is set the api format requires it. (always for xml)
+        * including setupExternalResponse where the printer is set.
         * The module is then executed and results printed.
         */
        private function executeApiModule( ApiBase $module ) {
diff --git a/repo/tests/phpunit/includes/api/ModifyTermTestCase.php 
b/repo/tests/phpunit/includes/api/ModifyTermTestCase.php
index e33f883..0899b44 100644
--- a/repo/tests/phpunit/includes/api/ModifyTermTestCase.php
+++ b/repo/tests/phpunit/includes/api/ModifyTermTestCase.php
@@ -114,15 +114,11 @@
 
                // -- check item in database 
-------------------------------------------
                $dbEntity = $this->loadEntity( EntityTestHelper::getId( 'Empty' 
) );
-               if ( count( $expected['value'] ) ) {
-                       $this->assertArrayHasKey( $attribute, $dbEntity );
-                       $dbLabels = $this->flattenArray( $dbEntity[$attribute], 
'language', 'value', true );
-                       foreach ( $expected['value'] as $valueLanguage => 
$value ) {
-                               $this->assertArrayHasKey( $valueLanguage, 
$dbLabels );
-                               $this->assertEquals( $value, 
$dbLabels[$valueLanguage][0] );
-                       }
-               } else {
-                       $this->assertArrayNotHasKey( $attribute, $dbEntity );
+               $this->assertArrayHasKey( $attribute, $dbEntity );
+               $dbLabels = $this->flattenArray( $dbEntity[$attribute], 
'language', 'value', true );
+               foreach ( $expected['value'] as $valueLanguage => $value ) {
+                       $this->assertArrayHasKey( $valueLanguage, $dbLabels );
+                       $this->assertEquals( $value, 
$dbLabels[$valueLanguage][0] );
                }
 
                // -- check the edit summary 
--------------------------------------------
diff --git a/repo/tests/phpunit/includes/api/ResultBuilderTest.php 
b/repo/tests/phpunit/includes/api/ResultBuilderTest.php
index 8653af6..465f5fe 100644
--- a/repo/tests/phpunit/includes/api/ResultBuilderTest.php
+++ b/repo/tests/phpunit/includes/api/ResultBuilderTest.php
@@ -44,7 +44,7 @@
                return new ApiResult( false );
        }
 
-       private function getResultBuilder( $result, $isRawMode = false ) {
+       private function getResultBuilder( $result, $addMetaData = false ) {
                $mockTitle = $this->getMockBuilder( '\Title' )
                        ->disableOriginalConstructor()
                        ->getMock();
@@ -82,10 +82,31 @@
                        $serializerFactory,
                        new MockSiteStore(),
                        $mockPropertyDataTypeLookup,
-                       $isRawMode
+                       $addMetaData
                );
 
                return $builder;
+       }
+
+       /**
+        * Removes all metadata keys as recognised by the MW Api.
+        * These all start with a '_' character.
+        *
+        * @param array $array
+        *
+        * @return array
+        */
+       private function removeMetaData( array $array ) {
+               foreach ( $array as $key => &$value ) {
+                       if ( substr( $key, 0, 1 ) === '_' ) {
+                               unset( $array[$key] );
+                       } else {
+                               if ( is_array( $value ) ) {
+                                       $value = $this->removeMetaData( $value 
);
+                               }
+                       }
+               }
+               return $array;
        }
 
        public function testCanConstruct() {
@@ -103,8 +124,13 @@
                $resultBuilder->markSuccess( $param );
                $data = $result->getResultData();
 
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
-               $this->assertEquals( array( 'success' => $expected ), $data );
+               $this->assertEquals(
+                       array(
+                               'success' => $expected,
+                               '_type' => 'assoc',
+                       ),
+                       $data
+               );
        }
 
        public function provideMarkResultSuccess() {
@@ -150,19 +176,24 @@
                                                'en' => array(
                                                        array(
                                                                'language' => 
'en',
-                                                               'value' => 'bar'
+                                                               'value' => 
'bar',
                                                        ),
                                                        array(
                                                                'language' => 
'en',
-                                                               'value' => 'baz'
-                                                       )
+                                                               'value' => 
'baz',
+                                                       ),
+                                                       '_element' => 'alias',
                                                ),
                                                'zh' => array(
                                                        array(
                                                                'language' => 
'zh',
                                                                'value' => 
'????????',
                                                        ),
+                                                       '_element' => 'alias',
                                                ),
+                                               '_element' => 'language',
+                                               '_type' => 'kvp',
+                                               '_kvpkeyname' => 'id',
                                        ),
                                        'descriptions' => array(
                                                'pt' => array(
@@ -173,6 +204,8 @@
                                                        'language' => 'pl',
                                                        'value' => 'Longer 
Description For An Item'
                                                ),
+                                               '_element' => 'description',
+                                               '_type' => 'array',
                                        ),
                                        'labels' => array(
                                                'de' => array(
@@ -183,6 +216,8 @@
                                                        'language' => 
'zh_classical',
                                                        'value' => 'Longer 
Label'
                                                ),
+                                               '_element' => 'label',
+                                               '_type' => 'array',
                                        ),
                                        'claims' => array(
                                                'P65' => array(
@@ -216,145 +251,11 @@
                                                                                
        'property' => 'P65',
                                                                                
        'datatype' => 'DtIdFor_P65',
                                                                                
),
-                                                                       ),
-                                                               ),
-                                                               'rank' => 
'normal',
-                                                               
'qualifiers-order' => array(
-                                                                       'P65'
-                                                               ),
-                                                               'references' => 
array(
-                                                                       array(
-                                                                               
'hash' => 'bdc5f7185904d6d3219e13b7443571dda8c4bee8',
-                                                                               
'snaks' => array(
-                                                                               
        'P65' => array(
-                                                                               
                array(
-                                                                               
                        'snaktype' => 'somevalue',
-                                                                               
                        'property' => 'P65',
-                                                                               
                        'datatype' => 'DtIdFor_P65',
-                                                                               
                )
-                                                                               
        ),
-                                                                               
        'P68' => array(
-                                                                               
                array(
-                                                                               
                        'snaktype' => 'somevalue',
-                                                                               
                        'property' => 'P68',
-                                                                               
                        'datatype' => 'DtIdFor_P68',
-                                                                               
                )
-                                                                               
        ),
-                                                                               
),
-                                                                               
'snaks-order' => array(
-                                                                               
        'P65',
-                                                                               
        'P68'
-                                                                               
)
-                                                                       ),
-                                                               ),
-                                                       )
-                                               ),
-                                       ),
-                                       'sitelinks' => array(
-                                               'enwiki' => array(
-                                                       'site' => 'enwiki',
-                                                       'title' => 'Berlin',
-                                                       'badges' => array( 
'Q333' )
-                                               ),
-                                               'zh_classicalwiki' => array(
-                                                       'site' => 
'zh_classicalwiki',
-                                                       'title' => 
'User:Addshore',
-                                                       'badges' => array()
-                                               ),
-                                       )
-                               ),
-                               '_element' => 'entity',
-                       ),
-               );
-
-               $expectedRaw = array(
-                       'entities' => array(
-                               array(
-                                       'pageid' => 123, //mocked
-                                       'ns' => 456, //mocked
-                                       'title' => 'MockPrefixedText', //mocked
-                                       'id' => 'Q123098',
-                                       'type' => 'item',
-                                       'lastrevid' => 33,
-                                       'modified' => '2013-11-26T20:29:23Z',
-                                       'redirects' => array(
-                                               'from' => 'Q1230000',
-                                               'to' => 'Q123098',
-                                       ),
-                                       'aliases' => array(
-                                               array(
-                                                       'language' => 'en',
-                                                       'value' => 'bar'
-                                               ),
-                                               array(
-                                                       'language' => 'en',
-                                                       'value' => 'baz'
-                                               ),
-                                               array(
-                                                       'language' => 'zh',
-                                                       'value' => '????????',
-                                               ),
-                                               '_element' => 'alias',
-                                       ),
-                                       'descriptions' => array(
-                                               array(
-                                                       'language' => 'pt',
-                                                       'value' => 'ptDesc'
-                                               ),
-                                               array(
-                                                       'language' => 'pl',
-                                                       'value' => 'Longer 
Description For An Item'
-                                               ),
-                                               '_element' => 'description',
-                                       ),
-                                       'labels' => array(
-                                               array(
-                                                       'language' => 'de',
-                                                       'value' => 'foo'
-                                               ),
-                                               array(
-                                                       'language' => 
'zh_classical',
-                                                       'value' => 'Longer 
Label'
-                                               ),
-                                               '_element' => 'label',
-                                       ),
-                                       'claims' => array(
-                                               array(
-                                                       'id' => 'P65',
-                                                       array(
-                                                               'id' => 
'imaguid',
-                                                               'mainsnak' => 
array(
-                                                                       
'snaktype' => 'value',
-                                                                       
'property' => 'P65',
-                                                                       
'datavalue' => array(
-                                                                               
'value' => 'snakStringValue',
-                                                                               
'type' => 'string',
-                                                                       ),
-                                                                       
'datatype' => 'DtIdFor_P65',
-                                                               ),
-                                                               'type' => 
'statement',
-                                                               'qualifiers' => 
array(
-                                                                       array(
-                                                                               
'id' => 'P65',
-                                                                               
array(
-                                                                               
        'hash' => 'e95e866e7fa1c18bd06dae9b712cb99545107eb8',
-                                                                               
        'snaktype' => 'value',
-                                                                               
        'property' => 'P65',
-                                                                               
        'datavalue' => array(
-                                                                               
                'value' => 'string!',
-                                                                               
                'type' => 'string',
-                                                                               
        ),
-                                                                               
        'datatype' => 'DtIdFor_P65',
-                                                                               
),
-                                                                               
array(
-                                                                               
        'hash' => '210b00274bf03247a89de918f15b12142ebf9e56',
-                                                                               
        'snaktype' => 'somevalue',
-                                                                               
        'property' => 'P65',
-                                                                               
        'datatype' => 'DtIdFor_P65',
-                                                                               
),
                                                                                
'_element' => 'qualifiers',
                                                                        ),
                                                                        
'_element' => 'property',
+                                                                       '_type' 
=> 'kvp',
+                                                                       
'_kvpkeyname' => 'id',
                                                                ),
                                                                'rank' => 
'normal',
                                                                
'qualifiers-order' => array(
@@ -365,8 +266,7 @@
                                                                        array(
                                                                                
'hash' => 'bdc5f7185904d6d3219e13b7443571dda8c4bee8',
                                                                                
'snaks' => array(
-                                                                               
        array(
-                                                                               
                'id' => 'P65',
+                                                                               
        'P65' => array(
                                                                                
                array(
                                                                                
                        'snaktype' => 'somevalue',
                                                                                
                        'property' => 'P65',
@@ -374,8 +274,7 @@
                                                                                
                ),
                                                                                
                '_element' => 'snak',
                                                                                
        ),
-                                                                               
        array(
-                                                                               
                'id' => 'P68',
+                                                                               
        'P68' => array(
                                                                                
                array(
                                                                                
                        'snaktype' => 'somevalue',
                                                                                
                        'property' => 'P68',
@@ -384,6 +283,8 @@
                                                                                
                '_element' => 'snak',
                                                                                
        ),
                                                                                
        '_element' => 'property',
+                                                                               
        '_type' => 'kvp',
+                                                                               
        '_kvpkeyname' => 'id',
                                                                                
),
                                                                                
'snaks-order' => array(
                                                                                
        'P65',
@@ -397,9 +298,11 @@
                                                        '_element' => 'claim',
                                                ),
                                                '_element' => 'property',
+                                               '_type' => 'kvp',
+                                               '_kvpkeyname' => 'id',
                                        ),
                                        'sitelinks' => array(
-                                               array(
+                                               'enwiki' => array(
                                                        'site' => 'enwiki',
                                                        'title' => 'Berlin',
                                                        'badges' => array(
@@ -407,7 +310,7 @@
                                                                '_element' => 
'badge',
                                                        )
                                                ),
-                                               array(
+                                               'zh_classicalwiki' => array(
                                                        'site' => 
'zh_classicalwiki',
                                                        'title' => 
'User:Addshore',
                                                        'badges' => array(
@@ -415,22 +318,29 @@
                                                        )
                                                ),
                                                '_element' => 'sitelink',
-                                       )
+                                               '_type' => 'array',
+                                       ),
                                ),
                                '_element' => 'entity',
+                               '_type' => 'array',
                        ),
+                       '_type' => 'assoc',
                );
 
+               $expectedNoMetaData = $this->removeMetaData( $expected );
+               // The api always starts with this
+               $expectedNoMetaData['_type'] = 'assoc';
+
                return array(
-                       array( false, $expected ),
-                       array( true, $expectedRaw ),
+                       array( false, $expectedNoMetaData ),
+                       array( true, $expected ),
                );
        }
 
        /**
         * @dataProvider provideTestAddEntityRevision
         */
-       public function testAddEntityRevision( $isRawMode, $expected ) {
+       public function testAddEntityRevision( $addMetaData, $expected ) {
                $result = $this->getDefaultResult();
                $item = new Item( new ItemId( 'Q123098' ) );
 
@@ -461,12 +371,11 @@
 
                $entityRevision = new EntityRevision( $item, 33, 
'20131126202923' );
 
-               $resultBuilder = $this->getResultBuilder( $result, $isRawMode );
+               $resultBuilder = $this->getResultBuilder( $result, $addMetaData 
);
                $resultBuilder->addEntityRevision( 'Q1230000', $entityRevision 
);
 
                $data = $result->getResultData();
 
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
                $this->assertEquals( $expected, $data );
        }
 
@@ -493,116 +402,74 @@
        }
 
        public function provideTestAddEntityRevisionFallback() {
+               $expected = array(
+                       'entities' => array(
+                               'Q123101' => array(
+                                       'id' => 'Q123101',
+                                       'type' => 'item',
+                                       'labels' => array(
+                                               'de-formal' => array(
+                                                       'language' => 'de',
+                                                       'value' => 'Oslo-de',
+                                                       'for-language' => 
'de-formal'
+                                               ),
+                                               'es' => array(
+                                                       'language' => 'en',
+                                                       'value' => 'Oslo-en',
+                                                       'for-language' => 'es',
+                                               ),
+                                               'qug' => array(
+                                                       'language' => 'en',
+                                                       'value' => 'Oslo-en',
+                                                       'for-language' => 'qug'
+                                               ),
+                                               'zh-my' => array(
+                                                       'language' => 'en',
+                                                       'value' => 'Oslo-en',
+                                                       'for-language' => 
'zh-my'
+                                               ),
+                                               '_element' => 'label',
+                                               '_type' => 'array',
+                                       ),
+                                       'descriptions' => array(
+                                               'es' => array(
+                                                       'language' => 'es',
+                                                       'value' => 'desc-es',
+                                               ),
+                                               'qug' => array(
+                                                       'language' => 'es',
+                                                       'value' => 'desc-es',
+                                                       'for-language' => 'qug'
+                                               ),
+                                               'zh-my' => array(
+                                                       'language' => 'zh-my',
+                                                       'value' => 'desc-zh-sg',
+                                                       'source-language' => 
'zh-sg',
+                                               ),
+                                               '_element' => 'description',
+                                               '_type' => 'array',
+                                       ),
+                               ),
+                               '_element' => 'entity',
+                               '_type' => 'array',
+                       ),
+                       '_type' => 'assoc',
+               );
+
+               $expectedNoMetaData = $this->removeMetaData( $expected );
+               // The api always starts with this
+               $expectedNoMetaData['_type'] = 'assoc';
+
                return array(
-                       array(
-                               false,
-                               array(
-                                       'entities' => array(
-                                               'Q123101' => array(
-                                                       'id' => 'Q123101',
-                                                       'type' => 'item',
-                                                       'labels' => array(
-                                                               'de-formal' => 
array(
-                                                                       
'language' => 'de',
-                                                                       'value' 
=> 'Oslo-de',
-                                                                       
'for-language' => 'de-formal',
-                                                               ),
-                                                               'es' => array(
-                                                                       
'language' => 'en',
-                                                                       'value' 
=> 'Oslo-en',
-                                                                       
'for-language' => 'es',
-                                                               ),
-                                                               'qug' => array(
-                                                                       
'language' => 'en',
-                                                                       'value' 
=> 'Oslo-en',
-                                                                       
'for-language' => 'qug',
-                                                               ),
-                                                               'zh-my' => 
array(
-                                                                       
'language' => 'en',
-                                                                       'value' 
=> 'Oslo-en',
-                                                                       
'for-language' => 'zh-my',
-                                                               ),
-                                                       ),
-                                                       'descriptions' => array(
-                                                               'es' => array(
-                                                                       
'language' => 'es',
-                                                                       'value' 
=> 'desc-es',
-                                                               ),
-                                                               'qug' => array(
-                                                                       
'language' => 'es',
-                                                                       'value' 
=> 'desc-es',
-                                                                       
'for-language' => 'qug',
-                                                               ),
-                                                               'zh-my' => 
array(
-                                                                       
'language' => 'zh-my',
-                                                                       'value' 
=> 'desc-zh-sg',
-                                                                       
'source-language' => 'zh-sg',
-                                                               ),
-                                                       ),
-                                               ),
-                                               '_element' => 'entity',
-                                       )
-                               ),
-                       ),
-                       array(
-                               true,
-                               array(
-                                       'entities' => array(
-                                               array(
-                                                       'id' => 'Q123101',
-                                                       'type' => 'item',
-                                                       'labels' => array(
-                                                               array(
-                                                                       
'language' => 'de',
-                                                                       'value' 
=> 'Oslo-de',
-                                                                       
'for-language' => 'de-formal'
-                                                               ),
-                                                               array(
-                                                                       
'language' => 'en',
-                                                                       'value' 
=> 'Oslo-en',
-                                                                       
'for-language' => 'es',
-                                                               ),
-                                                               array(
-                                                                       
'language' => 'en',
-                                                                       'value' 
=> 'Oslo-en',
-                                                                       
'for-language' => 'qug'
-                                                               ),
-                                                               array(
-                                                                       
'language' => 'en',
-                                                                       'value' 
=> 'Oslo-en',
-                                                                       
'for-language' => 'zh-my'
-                                                               ),
-                                                               '_element' => 
'label',
-                                                       ),
-                                                       'descriptions' => array(
-                                                               array(
-                                                                       
'language' => 'es',
-                                                                       'value' 
=> 'desc-es',
-                                                               ),
-                                                               array(
-                                                                       
'language' => 'es',
-                                                                       'value' 
=> 'desc-es',
-                                                                       
'for-language' => 'qug'
-                                                               ),
-                                                               array(
-                                                                       
'language' => 'zh-my',
-                                                                       'value' 
=> 'desc-zh-sg',
-                                                                       
'source-language' => 'zh-sg',
-                                                               ),
-                                                               '_element' => 
'description',
-                                                       ),
-                                               ),
-                                               '_element' => 'entity',
-                                       )
-                               ),
-                       ),
+                       array( false, $expectedNoMetaData ),
+                       array( true, $expected ),
                );
        }
 
        /**
         * @dataProvider provideTestAddEntityRevisionFallback
         */
-       public function testAddEntityRevisionFallback( $isRawMode, $expected ) {
+       public function testAddEntityRevisionFallback( $addMetaData, $expected 
) {
                $item = new Item( new ItemId( 'Q123101' ) );
                $item->getFingerprint()->setLabel( 'de', 'Oslo-de' );
                $item->getFingerprint()->setLabel( 'en', 'Oslo-en' );
@@ -624,7 +491,7 @@
                $filterLangCodes = array_keys( $fallbackChains );
 
                $result = $this->getDefaultResult();
-               $resultBuilder = $this->getResultBuilder( $result, $isRawMode );
+               $resultBuilder = $this->getResultBuilder( $result, $addMetaData 
);
                $resultBuilder->addEntityRevision(
                        null,
                        $entityRevision,
@@ -635,7 +502,6 @@
                );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
 
                $this->assertEquals( $expected, $data );
        }
@@ -660,36 +526,39 @@
                        array( 'de' )
                );
 
-               $expected = array( 'entities' => array(
-                       'Q123099' => array(
-                               'id' => 'Q123099',
-                               'type' => 'item',
-                               'labels' => array(
-                                       'de' => array(
-                                               'language' => 'de',
-                                               'value' => 'text',
-                                       ),
-                               ),
-                               'descriptions' => array(
-                                       'de' => array(
-                                               'language' => 'de',
-                                               'value' => 'text',
-                                       ),
-                               ),
-                               'aliases' => array(
-                                       'de' => array(
-                                               array(
+               $expected = array(
+                       'entities' => array(
+                               'Q123099' => array(
+                                       'id' => 'Q123099',
+                                       'type' => 'item',
+                                       'labels' => array(
+                                               'de' => array(
                                                        'language' => 'de',
                                                        'value' => 'text',
                                                ),
                                        ),
+                                       'descriptions' => array(
+                                               'de' => array(
+                                                       'language' => 'de',
+                                                       'value' => 'text',
+                                               ),
+                                       ),
+                                       'aliases' => array(
+                                               'de' => array(
+                                                       array(
+                                                               'language' => 
'de',
+                                                               'value' => 
'text',
+                                                       ),
+                                               ),
+                                       ),
                                ),
                        ),
-                       '_element' => 'entity',
-               ) );
+                       // This meta data element is always present in ApiResult
+                       '_type' => 'assoc',
+               );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
@@ -706,23 +575,26 @@
                $resultBuilder = $this->getResultBuilder( $result );
                $resultBuilder->addEntityRevision( null, $entityRevision, 
$props, $siteIds );
 
-               $expected = array( 'entities' => array(
-                       'Q123099' => array(
-                               'id' => 'Q123099',
-                               'type' => 'item',
-                               'sitelinks' => array(
-                                       'enwiki' => array(
-                                               'site' => 'enwiki',
-                                               'title' => 'Berlin',
-                                               'badges' => array()
+               $expected = array(
+                       'entities' => array(
+                               'Q123099' => array(
+                                       'id' => 'Q123099',
+                                       'type' => 'item',
+                                       'sitelinks' => array(
+                                               'enwiki' => array(
+                                                       'site' => 'enwiki',
+                                                       'title' => 'Berlin',
+                                                       'badges' => array(),
+                                               ),
                                        ),
                                ),
                        ),
-                       '_element' => 'entity',
-               ) );
+                       // This meta data element is always present in ApiResult
+                       '_type' => 'assoc',
+               );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
@@ -742,42 +614,50 @@
                $resultBuilder = $this->getResultBuilder( $result, true );
                $resultBuilder->addEntityRevision( null, $entityRevision, 
$props, $siteIds );
 
-               $expected = array( 'entities' => array(
-                       array(
-                               'id' => 'Q123100',
-                               'type' => 'item',
-                               'sitelinks' => array(
-                                       array(
-                                               'site' => 'enwiki',
-                                               'title' => 'Berlin',
-                                               'badges' => array(
-                                                       '_element' => 'badge'
-                                               )
+               $expected = array(
+                       'entities' => array(
+                               'Q123100' => array(
+                                       'id' => 'Q123100',
+                                       'type' => 'item',
+                                       'sitelinks' => array(
+                                               'enwiki' => array(
+                                                       'site' => 'enwiki',
+                                                       'title' => 'Berlin',
+                                                       'badges' => array(
+                                                               '_element' => 
'badge',
+                                                       ),
+                                               ),
+                                               '_element' => 'sitelink',
+                                               '_type' => 'array',
                                        ),
-                                       '_element' => 'sitelink'
                                ),
+                               '_element' => 'entity',
+                               '_type' => 'array',
                        ),
-                       '_element' => 'entity'
-               ) );
+                       '_type' => 'assoc',
+               );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
        public function testAddBasicEntityInformation() {
                $result = $this->getDefaultResult();
                $entityId = new ItemId( 'Q67' );
-               $expected = array( 'entity' => array(
-                       'id' => 'Q67',
-                       'type' => 'item',
-               ) );
+               $expected = array(
+                       'entity' => array(
+                               'id' => 'Q67',
+                               'type' => 'item',
+                       ),
+                       '_type' => 'assoc',
+               );
 
                $resultBuilder = $this->getResultBuilder( $result );
                $resultBuilder->addBasicEntityInformation( $entityId, 'entity' 
);
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
@@ -803,13 +683,14 @@
                                        ),
                                ),
                        ),
+                       '_type' => 'assoc',
                );
 
                $resultBuilder = $this->getResultBuilder( $result );
                $resultBuilder->addLabels( $labels, $path );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
@@ -827,13 +708,14 @@
                                        ),
                                ),
                        ),
+                       '_type' => 'assoc',
                );
 
                $resultBuilder = $this->getResultBuilder( $result );
                $resultBuilder->addRemovedLabel( 'en', $path );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
@@ -859,13 +741,14 @@
                                        ),
                                ),
                        ),
+                       '_type' => 'assoc',
                );
 
                $resultBuilder = $this->getResultBuilder( $result );
                $resultBuilder->addDescriptions( $descriptions, $path );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
@@ -883,84 +766,67 @@
                                        ),
                                ),
                        ),
+                       '_type' => 'assoc',
                );
 
                $resultBuilder = $this->getResultBuilder( $result );
                $resultBuilder->addRemovedDescription( 'en', $path );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
        public function provideAddAliasGroupList() {
+               $expected = array(
+                       'entities' => array(
+                               'Q1' => array(
+                                       'aliases' => array(
+                                               'en' => array(
+                                                       array(
+                                                               'language' => 
'en',
+                                                               'value' => 
'boo',
+                                                       ),
+                                                       array(
+                                                               'language' => 
'en',
+                                                               'value' => 
'hoo',
+                                                       ),
+                                                       '_element' => 'alias',
+                                               ),
+                                               'de' => array(
+                                                       array(
+                                                               'language' => 
'de',
+                                                               'value' => 
'ham',
+                                                       ),
+                                                       array(
+                                                               'language' => 
'de',
+                                                               'value' => 
'cheese',
+                                                       ),
+                                                       '_element' => 'alias',
+                                               ),
+                                               '_element' => 'language',
+                                               '_type' => 'kvp',
+                                               '_kvpkeyname' => 'id',
+                                       ),
+                               ),
+                       ),
+                       '_type' => 'assoc',
+               );
+
+               $expectedNoMetaData = $this->removeMetaData( $expected );
+               // The api always starts with this
+               $expectedNoMetaData['_type'] = 'assoc';
+
                return array(
-                       array(
-                               false,
-                               array(
-                                       'entities' => array(
-                                               'Q1' => array(
-                                                       'aliases' => array(
-                                                               'en' => array(
-                                                                       array(
-                                                                               
'language' => 'en',
-                                                                               
'value' => 'boo',
-                                                                       ),
-                                                                       array(
-                                                                               
'language' => 'en',
-                                                                               
'value' => 'hoo',
-                                                                       ),
-                                                               ),
-                                                               'de' => array(
-                                                                       array(
-                                                                               
'language' => 'de',
-                                                                               
'value' => 'ham',
-                                                                       ),
-                                                                       array(
-                                                                               
'language' => 'de',
-                                                                               
'value' => 'cheese',
-                                                                       ),
-                                                               ),
-                                                       ),
-                                               ),
-                                       ),
-                               ),
-                       ),
-                       array(
-                               true,
-                               array(
-                                       'entities' => array(
-                                               'Q1' => array(
-                                                       'aliases' => array(
-                                                               array(
-                                                                       
'language' => 'en',
-                                                                       'value' 
=> 'boo',
-                                                               ),
-                                                               array(
-                                                                       
'language' => 'en',
-                                                                       'value' 
=> 'hoo',
-                                                               ),
-                                                               array(
-                                                                       
'language' => 'de',
-                                                                       'value' 
=> 'ham',
-                                                               ),
-                                                               array(
-                                                                       
'language' => 'de',
-                                                                       'value' 
=> 'cheese',
-                                                               ),
-                                                               '_element' => 
'alias',
-                                                       ),
-                                               ),
-                                       ),
-                               ),
-                       ),
+                       array( false, $expectedNoMetaData ),
+                       array( true, $expected ),
                );
        }
 
        /**
         * @dataProvider provideAddAliasGroupList
         */
-       public function testAddAliasGroupList( $rawMode, $expected ) {
+       public function testAddAliasGroupList( $metaData, $expected ) {
                $result = $this->getDefaultResult();
                $aliasGroupList = new AliasGroupList(
                        array(
@@ -970,66 +836,51 @@
                );
                $path = array( 'entities', 'Q1' );
 
-               $resultBuilder = $this->getResultBuilder( $result, $rawMode );
+               $resultBuilder = $this->getResultBuilder( $result, $metaData );
                $resultBuilder->addAliasGroupList( $aliasGroupList, $path );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
        public function provideAddSiteLinkList() {
+               $expected = array(
+                       'entities' => array(
+                               'Q1' => array(
+                                       'sitelinks' => array(
+                                               'enwiki' => array(
+                                                       'site' => 'enwiki',
+                                                       'title' => 
'User:Addshore',
+                                                       'badges' => array( 
'_element' => 'badge' ),
+                                               ),
+                                               'dewikivoyage' => array(
+                                                       'site' => 
'dewikivoyage',
+                                                       'title' => 'Berlin',
+                                                       'badges' => array( 
'_element' => 'badge' ),
+                                               ),
+                                               '_element' => 'sitelink',
+                                               '_type' => 'array',
+                                       ),
+                               ),
+                       ),
+                       '_type' => 'assoc',
+               );
+
+               $expectedNoMetaData = $this->removeMetaData( $expected );
+               // The api always starts with this
+               $expectedNoMetaData['_type'] = 'assoc';
+
                return array(
-                       array(
-                               false,
-                               array(
-                                       'entities' => array(
-                                               'Q1' => array(
-                                                       'sitelinks' => array(
-                                                               'enwiki' => 
array(
-                                                                       'site' 
=> 'enwiki',
-                                                                       'title' 
=> 'User:Addshore',
-                                                                       
'badges' => array(),
-                                                               ),
-                                                               'dewikivoyage' 
=> array(
-                                                                       'site' 
=> 'dewikivoyage',
-                                                                       'title' 
=> 'Berlin',
-                                                                       
'badges' => array(),
-                                                               ),
-                                                       ),
-                                               ),
-                                       ),
-                               ),
-                       ),
-                       array(
-                               true,
-                               array(
-                                       'entities' => array(
-                                               'Q1' => array(
-                                                       'sitelinks' => array(
-                                                               array(
-                                                                       'site' 
=> 'enwiki',
-                                                                       'title' 
=> 'User:Addshore',
-                                                                       
'badges' => array( '_element' => 'badge' ),
-                                                               ),
-                                                               array(
-                                                                       'site' 
=> 'dewikivoyage',
-                                                                       'title' 
=> 'Berlin',
-                                                                       
'badges' => array( '_element' => 'badge' ),
-                                                               ),
-                                                               '_element' => 
'sitelink',
-                                                       ),
-                                               ),
-                                       ),
-                               ),
-                       ),
+                       array( false, $expectedNoMetaData ),
+                       array( true, $expected ),
                );
        }
 
        /**
         * @dataProvider provideAddSiteLinkList
         */
-       public function testAddSiteLinkList( $isRawMode, $expected ) {
+       public function testAddSiteLinkList( $addMetaData, $expected ) {
                $result = $this->getDefaultResult();
                $siteLinkList = new SiteLinkList(
                        array(
@@ -1039,15 +890,16 @@
                );
                $path = array( 'entities', 'Q1' );
 
-               $resultBuilder = $this->getResultBuilder( $result, $isRawMode );
+               $resultBuilder = $this->getResultBuilder( $result, $addMetaData 
);
                $resultBuilder->addSiteLinkList( $siteLinkList, $path );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
        public function testAddRemovedSiteLinks() {
+               //TODO test with metadata....
                $result = $this->getDefaultResult();
                $siteLinkList = new SiteLinkList( array(
                        new SiteLink( 'enwiki', 'User:Addshore' ),
@@ -1073,16 +925,14 @@
                                        ),
                                ),
                        ),
+                       '_type' => 'assoc',
                );
 
                $resultBuilder = $this->getResultBuilder( $result );
                $resultBuilder->addRemovedSiteLinks( $siteLinkList, $path );
 
-               $data = $result->getResultData( null, array(
-                       'BC' => array(),
-                       'Types' => array(),
-                       'Strip' => 'all',
-               ) );
+               $data = $result->getResultData();
+
                $this->assertEquals( $expected, $data );
        }
 
@@ -1128,6 +978,7 @@
                                        ),
                                ),
                        ),
+                       '_type' => 'assoc',
                );
 
                $resultBuilder = $this->getResultBuilder( $result );
@@ -1135,55 +986,48 @@
                $resultBuilder->addRemovedSiteLinks( $siteLinkListRemove, $path 
);
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
        /**
         * @dataProvider statementSerializationProvider
         */
-       public function testAddClaims( Statement $statement, $isRawMode, 
$statementSerialization ) {
+       public function testAddStatements( Statement $statement, $addMetaData, 
$statementSerialization ) {
                $result = $this->getDefaultResult();
                $path = array( 'entities', 'Q1' );
 
-               if ( $isRawMode ) {
-                       $expected = array(
-                               'entities' => array(
-                                       'Q1' => array(
-                                               'claims' => array(
-                                                       array(
-                                                               'id' => 'P12',
-                                                               
$statementSerialization,
-                                                               '_element' => 
'claim',
-                                                       ),
-                                                       '_element' => 
'property',
+               $expected = array(
+                       'entities' => array(
+                               'Q1' => array(
+                                       'claims' => array(
+                                               'P12' => array(
+                                                       $statementSerialization,
+                                                       '_element' => 'claim',
                                                ),
+                                               '_element' => 'property',
+                                               '_type' => 'kvp',
+                                               '_kvpkeyname' => 'id',
                                        ),
                                ),
-                       );
-               } else {
-                       $expected = array(
-                               'entities' => array(
-                                       'Q1' => array(
-                                               'claims' => array(
-                                                       'P12' => array(
-                                                               
$statementSerialization
-                                                       ),
-                                               ),
-                                       ),
-                               ),
-                       );
+                       ),
+                       '_type' => 'assoc',
+               );
+
+               if ( !$addMetaData ) {
+                       $expected = $this->removeMetaData( $expected );
+                       $expected['_type'] = 'assoc';
                }
 
-               $resultBuilder = $this->getResultBuilder( $result, $isRawMode );
+               $resultBuilder = $this->getResultBuilder( $result, $addMetaData 
);
                $resultBuilder->addStatements( array( $statement ), $path );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
-       public function testAddClaimsNoProps() {
+       public function testAddStatementsNoProps() {
                $result = $this->getDefaultResult();
                $path = array( 'entities', 'Q1' );
 
@@ -1217,6 +1061,7 @@
                                        ),
                                ),
                        ),
+                       '_type' => 'assoc',
                );
 
                $props = array();
@@ -1225,22 +1070,25 @@
                $resultBuilder->addStatements( array( $statement ), $path, 
$props );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
        /**
         * @dataProvider statementSerializationProvider
         */
-       public function testAddClaim( Statement $statement, $isRawMode, 
$statementSerialization ) {
+       public function testAddStatement( Statement $statement, $addMetaData, 
$statementSerialization ) {
                $result = $this->getDefaultResult();
-               $expected = array( 'claim' => $statementSerialization );
+               $expected = array(
+                       'claim' => $statementSerialization,
+                       '_type' => 'assoc',
+               );
 
-               $resultBuilder = $this->getResultBuilder( $result, $isRawMode );
+               $resultBuilder = $this->getResultBuilder( $result, $addMetaData 
);
                $resultBuilder->addStatement( $statement );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
@@ -1258,56 +1106,7 @@
                        'fooguidbar'
                );
 
-               $expectedSerialization = array(
-                       'id' => 'fooguidbar',
-                       'mainsnak' => array(
-                               'snaktype' => 'value',
-                               'property' => 'P12',
-                               'datavalue' => array(
-                                       'value' => 'stringVal',
-                                       'type' => 'string',
-                               ),
-                               'datatype' => 'DtIdFor_P12',
-                       ),
-                       'type' => 'statement',
-                       'rank' => 'normal',
-                       'qualifiers-order' => array( 'P12' ),
-                       'references' => array(
-                               array(
-                                       'hash' => 
'2f543336756784850a310cbc52a9307e467c7c42',
-                                       'snaks' => array(
-                                               'P12' => array(
-                                                       array(
-                                                               'snaktype' => 
'value',
-                                                               'property' => 
'P12',
-                                                               'datatype' => 
'DtIdFor_P12',
-                                                               'datavalue' => 
array(
-                                                                       'value' 
=> 'refSnakVal',
-                                                                       'type' 
=> 'string',
-                                                               ),
-                                                       ),
-                                               ),
-                                       ),
-                                       'snaks-order' => array( 'P12' ),
-                               ),
-                       ),
-                       'qualifiers' => array(
-                               'P12' => array(
-                                       array(
-                                               'snaktype' => 'value',
-                                               'property' => 'P12',
-                                               'datatype' => 'DtIdFor_P12',
-                                               'datavalue' => array(
-                                                       'value' => 
'qualiferVal',
-                                                       'type' => 'string',
-                                               ),
-                                               'hash' => 
'67423e8a140238decaa9156be1e3ba23513b3b19',
-                                       ),
-                               ),
-                       ),
-               );
-
-               $expectedRawModeSerialization = array(
+               $expected = array(
                        'id' => 'fooguidbar',
                        'mainsnak' => array(
                                'snaktype' => 'value',
@@ -1328,8 +1127,7 @@
                                array(
                                        'hash' => 
'2f543336756784850a310cbc52a9307e467c7c42',
                                        'snaks' => array(
-                                               array(
-                                                       'id' => 'P12',
+                                               'P12' => array(
                                                        array(
                                                                'snaktype' => 
'value',
                                                                'property' => 
'P12',
@@ -1342,6 +1140,8 @@
                                                        '_element' => 'snak',
                                                ),
                                                '_element' => 'property',
+                                               '_type' => 'kvp',
+                                               '_kvpkeyname' => 'id',
                                        ),
                                        'snaks-order' => array(
                                                'P12',
@@ -1351,8 +1151,7 @@
                                '_element' => 'reference',
                        ),
                        'qualifiers' => array(
-                               array(
-                                       'id' => 'P12',
+                               'P12' => array(
                                        array(
                                                'snaktype' => 'value',
                                                'property' => 'P12',
@@ -1366,74 +1165,62 @@
                                        '_element' => 'qualifiers',
                                ),
                                '_element' => 'property',
+                               '_type' => 'kvp',
+                               '_kvpkeyname' => 'id',
                        ),
                );
 
+               $expectedNoMetaData = $this->removeMetaData( $expected );
+
                return array(
-                       array( $statement, false, $expectedSerialization ),
-                       array( $statement, true, $expectedRawModeSerialization 
),
+                       array( $statement, false, $expectedNoMetaData ),
+                       array( $statement, true, $expected ),
                );
        }
 
        public function provideAddReference() {
+               $expected = array(
+                       'reference' => array(
+                               'hash' => 
'de52176deca6dd967b2a4122c96766089c1f793a',
+                               'snaks' => array(
+                                       'P12' => array(
+                                               array(
+                                                       'snaktype' => 'value',
+                                                       'property' => 'P12',
+                                                       'datavalue' => array(
+                                                               'value' => 
'stringVal',
+                                                               'type' => 
'string',
+                                                       ),
+                                                       'datatype' => 
'DtIdFor_P12',
+                                               ),
+                                               '_element' => 'snak',
+                                       ),
+                                       '_element' => 'property',
+                                       '_type' => 'kvp',
+                                       '_kvpkeyname' => 'id',
+                               ),
+                               'snaks-order' => array(
+                                       'P12',
+                                       '_element' => 'property',
+                               ),
+                       ),
+                       '_type' => 'assoc',
+               );
+
+               $expectedNoMetaData = $this->removeMetaData( $expected );
+               // The api always starts with this
+               $expectedNoMetaData['_type'] = 'assoc';
+
                return array(
-                       array(
-                               false,
-                               array(
-                                       'reference' => array(
-                                               'hash' => 
'de52176deca6dd967b2a4122c96766089c1f793a',
-                                               'snaks' => array(
-                                                       'P12' => array(
-                                                               array(
-                                                                       
'snaktype' => 'value',
-                                                                       
'property' => 'P12',
-                                                                       
'datavalue' => array(
-                                                                               
'value' => 'stringVal',
-                                                                               
'type' => 'string',
-                                                                       ),
-                                                                       
'datatype' => 'DtIdFor_P12',
-                                                               )
-                                                       ),
-                                               ),
-                                               'snaks-order' => array( 'P12' ),
-                                       ),
-                               )
-                       ),
-                       array(
-                               true,
-                               array(
-                                       'reference' => array(
-                                               'hash' => 
'de52176deca6dd967b2a4122c96766089c1f793a',
-                                               'snaks' => array(
-                                                       array(
-                                                               'id' => 'P12',
-                                                               array(
-                                                                       
'snaktype' => 'value',
-                                                                       
'property' => 'P12',
-                                                                       
'datavalue' => array(
-                                                                               
'value' => 'stringVal',
-                                                                               
'type' => 'string',
-                                                                       ),
-                                                                       
'datatype' => 'DtIdFor_P12',
-                                                               ),
-                                                               '_element' => 
'snak',
-                                                       ),
-                                                       '_element' => 
'property',
-                                               ),
-                                               'snaks-order' => array(
-                                                       'P12',
-                                                       '_element' => 
'property',
-                                               ),
-                                       ),
-                               )
-                       ),
+                       array( false, $expectedNoMetaData ),
+                       array( true, $expected ),
                );
        }
 
        /**
         * @dataProvider provideAddReference
         */
-       public function testAddReference( $isRawMode, $expected ) {
+       public function testAddReference( $addMetaData, $expected ) {
                $result = $this->getDefaultResult();
                $reference = new Reference(
                        new SnakList( array(
@@ -1441,20 +1228,20 @@
                        ) )
                );
 
-               $resultBuilder = $this->getResultBuilder( $result, $isRawMode );
+               $resultBuilder = $this->getResultBuilder( $result, $addMetaData 
);
                $resultBuilder->addReference( $reference );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
        /**
         * @dataProvider provideMissingEntity
         */
-       public function testAddMissingEntity( $missingEntities, $expected ) {
+       public function testAddMissingEntityWithMetaData( $missingEntities, 
$expected ) {
                $result = $this->getDefaultResult();
-               $resultBuilder = $this->getResultBuilder( $result );
+               $resultBuilder = $this->getResultBuilder( $result, true );
 
                foreach ( $missingEntities as $key => $missingDetails ) {
                        if ( is_int( $key ) ) {
@@ -1466,7 +1253,7 @@
                }
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
@@ -1485,6 +1272,7 @@
                                                ),
                                                '_element' => 'entity',
                                        ),
+                                       '_type' => 'assoc',
                                )
                        ),
                        array(
@@ -1499,6 +1287,7 @@
                                                ),
                                                '_element' => 'entity',
                                        ),
+                                       '_type' => 'assoc',
                                )
                        ),
                        array(
@@ -1513,6 +1302,7 @@
                                                ),
                                                '_element' => 'entity',
                                        ),
+                                       '_type' => 'assoc',
                                )
                        ),
                        array(
@@ -1534,6 +1324,7 @@
                                                ),
                                                '_element' => 'entity',
                                        ),
+                                       '_type' => 'assoc',
                                )
                        ),
                );
@@ -1551,13 +1342,14 @@
                                        'to' => 'Berlin'
                                ),
                        ),
+                       '_type' => 'assoc',
                );
 
                $resultBuilder = $this->getResultBuilder( $result );
                $resultBuilder->addNormalizedTitle( $from, $to );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
@@ -1573,47 +1365,61 @@
                $mockStatus->expects( $this->once() )
                        ->method( 'getValue' )
                        ->will( $this->returnValue( array( 'revision' => 
$mockRevision ) ) );
-               $expected = array( 'entity' => array( 'lastrevid' => '123' ) );
+               $expected = array(
+                       'entity' => array( 'lastrevid' => '123' ),
+                       '_type' => 'assoc',
+               );
 
                $resultBuilder = $this->getResultBuilder( $result );
                $resultBuilder->addRevisionIdFromStatusToResult( $mockStatus, 
'entity' );
 
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
+
                $this->assertEquals( $expected, $data );
        }
 
        public function provideSetList() {
                return array(
-                       'null path' => array( null, 'foo', array(), 'letter', 
false, array( 'foo' => array() ) ),
+                       'null path' => array( null, 'foo', array(), 'letter', 
false,
+                               array( 'foo' => array(), '_type' => 'assoc' )
+                       ),
                        'empty path' => array( array(), 'foo', array( 'x', 'y' 
), 'letter', false,
                                array(
-                                       'foo' => array( 'x', 'y' )
+                                       'foo' => array( 'x', 'y' ), '_type' => 
'assoc'
                                )
                        ),
                        'string path' => array( 'ROOT', 'foo', array( 'x', 'y' 
), 'letter', false,
                                array(
                                        'ROOT' => array(
-                                               'foo' => array( 'x', 'y' ) )
+                                               'foo' => array( 'x', 'y' )
+                                       ),
+                                       '_type' => 'assoc',
                                )
                        ),
                        'actual path' => array( array( 'one', 'two' ), 'foo', 
array( 'X' => 'x', 'Y' => 'y' ), 'letter', false,
                                array(
                                        'one' => array(
                                                'two' => array(
-                                                       'foo' => array( 'X' => 
'x', 'Y' => 'y' ) ) )
+                                                       'foo' => array( 'X' => 
'x', 'Y' => 'y' )
+                                               )
+                                       ),
+                                       '_type' => 'assoc',
                                )
                        ),
                        'indexed' => array( 'ROOT', 'foo', array( 'X' => 'x', 
'Y' => 'y' ), 'letter', true,
                                array(
                                        'ROOT' => array(
-                                               'foo' => array( 'x', 'y', 
'_element' => 'letter' ) )
-                               )
+                                               'foo' => array( 'X' => 'x', 'Y' 
=> 'y', '_element' => 'letter', '_type' => 'array' ),
+                                       ),
+                                       '_type' => 'assoc',
+                               ),
                        ),
                        'pre-set element name' => array( 'ROOT', 'foo', array( 
'x', 'y', '_element' => '_thingy' ), 'letter', true,
                                array(
                                        'ROOT' => array(
-                                               'foo' => array( 'x', 'y', 
'_element' => 'letter' ) )
+                                               'foo' => array( 'x', 'y', 
'_element' => '_thingy', '_type' => 'array' )
+                                       ),
+                                       '_type' => 'assoc',
                                )
                        ),
                );
@@ -1622,14 +1428,14 @@
        /**
         * @dataProvider provideSetList
         */
-       public function testSetList( $path, $name, array $values, $tag, 
$isRawMode, $expected ) {
+       public function testSetList( $path, $name, array $values, $tag, 
$addMetaData, $expected ) {
                $result = $this->getDefaultResult();
-               $builder = $this->getResultBuilder( $result, $isRawMode );
+               $builder = $this->getResultBuilder( $result, $addMetaData );
 
                $builder->setList( $path, $name, $values, $tag );
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
-               $this->assertResultStructure( $expected, $data );
+
+               $this->assertEquals( $expected, $data );
        }
 
        public function provideSetList_InvalidArgument() {
@@ -1657,27 +1463,33 @@
 
        public function provideSetValue() {
                return array(
-                       'null path' => array( null, 'foo', 'value', false, 
array( 'foo' => 'value' ) ),
+                       'null path' => array( null, 'foo', 'value', false, 
array( 'foo' => 'value', '_type' => 'assoc' ) ),
                        'empty path' => array( array(), 'foo', 'value', false,
                                array(
-                                       'foo' => 'value'
+                                       'foo' => 'value',
+                                       '_type' => 'assoc',
                                )
                        ),
                        'string path' => array( 'ROOT', 'foo', 'value', false,
                                array(
-                                       'ROOT' => array( 'foo' => 'value' )
+                                       'ROOT' => array( 'foo' => 'value' ),
+                                       '_type' => 'assoc'
                                )
                        ),
                        'actual path' => array( array( 'one', 'two' ), 'foo', 
array( 'X' => 'x', 'Y' => 'y' ), true,
                                array(
                                        'one' => array(
                                                'two' => array(
-                                                       'foo' => array( 'X' => 
'x', 'Y' => 'y' ) ) )
+                                                       'foo' => array( 'X' => 
'x', 'Y' => 'y' )
+                                               )
+                                       ),
+                                       '_type' => 'assoc'
                                )
                        ),
                        'indexed' => array( 'ROOT', 'foo', 'value', true,
                                array(
-                                       'ROOT' => array( 'foo' => 'value' )
+                                       'ROOT' => array( 'foo' => 'value' ),
+                                       '_type' => 'assoc'
                                )
                        ),
                );
@@ -1686,14 +1498,14 @@
        /**
         * @dataProvider provideSetValue
         */
-       public function testSetValue( $path, $name, $value, $isRawMode, 
$expected ) {
+       public function testSetValue( $path, $name, $value, $addMetaData, 
$expected ) {
                $result = $this->getDefaultResult();
-               $builder = $this->getResultBuilder( $result, $isRawMode );
+               $builder = $this->getResultBuilder( $result, $addMetaData );
 
                $builder->setValue( $path, $name, $value );
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
-               $this->assertResultStructure( $expected, $data );
+
+               $this->assertEquals( $expected, $data );
        }
 
        public function provideSetValue_InvalidArgument() {
@@ -1720,46 +1532,53 @@
        public function provideAppendValue() {
                return array(
                        'null path' => array( null, null, 'value', 'letter', 
false,
-                               array( 'value', '_element' => 'letter' ),
+                               array( 'value', '_type' => 'assoc' ),
                        ),
                        'empty path' => array( array(), null, 'value', 
'letter', false,
-                               array( 'value', '_element' => 'letter' )
+                               array( 'value', '_type' => 'assoc' )
                        ),
                        'string path' => array( 'ROOT', null, 'value', 
'letter', false,
                                array(
-                                       'ROOT' => array( 'value', '_element' => 
'letter' )
+                                       'ROOT' => array( 'value' ),
+                                       '_type' => 'assoc'
                                )
                        ),
                        'actual path' => array( array( 'one', 'two' ), null, 
array( 'X' => 'x', 'Y' => 'y' ), 'letter', false,
                                array(
                                        'one' => array(
-                                               'two' => array( array( 'X' => 
'x', 'Y' => 'y' ), '_element' => 'letter' ),
-                                       )
+                                               'two' => array( array( 'X' => 
'x', 'Y' => 'y' ) ),
+                                       ),
+                                       '_type' => 'assoc'
                                )
                        ),
                        'int key' => array( 'ROOT', -2, 'value', 'letter', 
false,
                                array(
-                                       'ROOT' => array( -2 => 'value', 
'_element' => 'letter' ),
+                                       'ROOT' => array( -2 => 'value' ),
+                                       '_type' => 'assoc',
                                )
                        ),
                        'string key' => array( 'ROOT', 'Q7', 'value', 'letter', 
false,
                                array(
-                                       'ROOT' => array( 'Q7' => 'value', 
'_element' => 'letter' ),
+                                       'ROOT' => array( 'Q7' => 'value' ),
+                                       '_type' => 'assoc',
                                )
                        ),
                        'null key indexed' => array( 'ROOT', null, 'value', 
'letter', true,
                                array(
-                                       'ROOT' => array( 'value', '_element' => 
'letter' )
+                                       'ROOT' => array( 'value', '_element' => 
'letter' ),
+                                       '_type' => 'assoc',
                                )
                        ),
                        'int key indexed' => array( 'ROOT', -2, 'value', 
'letter', true,
                                array(
-                                       'ROOT' => array( 'value', '_element' => 
'letter' )
+                                       'ROOT' => array( -2 => 'value', 
'_element' => 'letter' ),
+                                       '_type' => 'assoc',
                                )
                        ),
                        'string key indexed' => array( 'ROOT', 'Q7', 'value', 
'letter', true,
                                array(
-                                       'ROOT' => array( 'value', '_element' => 
'letter' )
+                                       'ROOT' => array( 'Q7' => 'value', 
'_element' => 'letter' ),
+                                       '_type' => 'assoc',
                                )
                        ),
                );
@@ -1768,14 +1587,14 @@
        /**
         * @dataProvider provideAppendValue
         */
-       public function testAppendValue( $path, $key, $value, $tag, $isRawMode, 
$expected ) {
+       public function testAppendValue( $path, $key, $value, $tag, 
$addMetaData, $expected ) {
                $result = $this->getDefaultResult();
-               $builder = $this->getResultBuilder( $result, $isRawMode );
+               $builder = $this->getResultBuilder( $result, $addMetaData );
 
                $builder->appendValue( $path, $key, $value, $tag );
                $data = $result->getResultData();
-               $this->removeElementsWithKeysRecursively( $data, array( '_type' 
) );
-               $this->assertResultStructure( $expected, $data );
+
+               $this->assertEquals( $expected, $data );
        }
 
        public function provideAppendValue_InvalidArgument() {
@@ -1798,34 +1617,6 @@
 
                $this->setExpectedException( 'InvalidArgumentException' );
                $builder->appendValue( $path, $key, $value, $tag );
-       }
-
-       private function assertResultStructure( $expected, $actual, $path = 
null ) {
-               foreach ( $expected as $key => $value ) {
-                       $this->assertArrayHasKey( $key, $actual, $path );
-
-                       if ( is_array( $value ) ) {
-                               $this->assertInternalType( 'array', 
$actual[$key], $path );
-
-                               $subKey = $path === null ? $key : $path . '/' . 
$key;
-                               $this->assertResultStructure( $value, 
$actual[$key], $subKey );
-                       } else {
-                               $this->assertEquals( $value, $actual[$key] );
-                       }
-               }
-
-               $this->assertEquals( array_keys( $expected ), array_keys( 
$actual ), "Keys of $path:" );
-       }
-
-       private function removeElementsWithKeysRecursively( array &$array, 
array $unwantedKeys ) {
-               foreach ( $unwantedKeys as $unwantedKey ) {
-                       unset( $array[$unwantedKey] );
-                       foreach ( $array as &$value ) {
-                               if ( is_array( $value ) ) {
-                                       
$this->removeElementsWithKeysRecursively( $value, array( $unwantedKey ) );
-                               }
-                       }
-               }
        }
 
 }
diff --git a/repo/tests/phpunit/includes/api/SetAliasesTest.php 
b/repo/tests/phpunit/includes/api/SetAliasesTest.php
index f46323e..70c8e01 100644
--- a/repo/tests/phpunit/includes/api/SetAliasesTest.php
+++ b/repo/tests/phpunit/includes/api/SetAliasesTest.php
@@ -118,14 +118,10 @@
 
                // -- check item in database 
-------------------------------------------
                $dbEntity = $this->loadEntity( EntityTestHelper::getId( 'Empty' 
) );
-               if ( count( $expected['value'] ) ) {
-                       $this->assertArrayHasKey( 'aliases', $dbEntity );
-                       $dbAliases = $this->flattenArray( $dbEntity['aliases'], 
'language', 'value', true );
-                       foreach ( $expected['value'] as $valueLanguage => 
$value ) {
-                               $this->assertArrayEquals( $value, $dbAliases[ 
$valueLanguage ] );
-                       }
-               } else {
-                       $this->assertArrayNotHasKey( 'aliases', $dbEntity );
+               $this->assertArrayHasKey( 'aliases', $dbEntity );
+               $dbAliases = $this->flattenArray( $dbEntity['aliases'], 
'language', 'value', true );
+               foreach ( $expected['value'] as $valueLanguage => $value ) {
+                       $this->assertArrayEquals( $value, $dbAliases[ 
$valueLanguage ] );
                }
 
                // -- check the edit summary 
--------------------------------------------
diff --git a/repo/tests/phpunit/includes/api/SetSiteLinkTest.php 
b/repo/tests/phpunit/includes/api/SetSiteLinkTest.php
index 522293a..206e1c6 100644
--- a/repo/tests/phpunit/includes/api/SetSiteLinkTest.php
+++ b/repo/tests/phpunit/includes/api/SetSiteLinkTest.php
@@ -495,13 +495,9 @@
                if ( array_key_exists( 'indb', $expected ) ) {
                        $expectedInDb = $expected['indb'];
                }
-               if ( $expectedInDb ) {
-                       $this->assertArrayHasKey( 'sitelinks', $dbEntity );
-                       $this->assertCount( $expectedInDb, 
$dbEntity['sitelinks'] );
-                       $this->assertContainsAllSiteLinks( $expected['value'], 
$dbEntity['sitelinks'] );
-               } else {
-                       $this->assertArrayNotHasKey( 'sitelinks', $dbEntity );
-               }
+               $this->assertArrayHasKey( 'sitelinks', $dbEntity );
+               $this->assertCount( $expectedInDb, $dbEntity['sitelinks'] );
+               $this->assertContainsAllSiteLinks( $expected['value'], 
$dbEntity['sitelinks'] );
 
                // -- check the edit summary 
--------------------------------------------
                if ( ! array_key_exists( 'warning', $expected ) || 
$expected['warning'] != 'edit-no-change' ) {

-- 
To view, visit https://gerrit.wikimedia.org/r/227686
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I6b8a041917304bb9c78153da5bd71327b83699c6
Gerrit-PatchSet: 25
Gerrit-Project: mediawiki/extensions/Wikibase
Gerrit-Branch: master
Gerrit-Owner: Addshore <addshorew...@gmail.com>
Gerrit-Reviewer: Addshore <addshorew...@gmail.com>
Gerrit-Reviewer: Anomie <bjor...@wikimedia.org>
Gerrit-Reviewer: Aude <aude.w...@gmail.com>
Gerrit-Reviewer: Bene <benestar.wikime...@gmail.com>
Gerrit-Reviewer: Daniel Kinzler <daniel.kinz...@wikimedia.de>
Gerrit-Reviewer: Hoo man <h...@online.de>
Gerrit-Reviewer: JanZerebecki <jan.wikime...@zerebecki.de>
Gerrit-Reviewer: Jeroen De Dauw <jeroended...@gmail.com>
Gerrit-Reviewer: Jonas Kress (WMDE) <jonas.kr...@wikimedia.de>
Gerrit-Reviewer: Legoktm <legoktm.wikipe...@gmail.com>
Gerrit-Reviewer: Lydia Pintscher <lydia.pintsc...@wikimedia.de>
Gerrit-Reviewer: Thiemo Mättig (WMDE) <thiemo.maet...@wikimedia.de>
Gerrit-Reviewer: jenkins-bot <>

_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

Reply via email to