Hello community, here is the log from the commit of package ghc-citeproc for openSUSE:Factory checked in at 2020-11-24 22:14:37 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/ghc-citeproc (Old) and /work/SRC/openSUSE:Factory/.ghc-citeproc.new.5913 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "ghc-citeproc" Tue Nov 24 22:14:37 2020 rev:4 rq:850494 version:0.2 Changes: -------- --- /work/SRC/openSUSE:Factory/ghc-citeproc/ghc-citeproc.changes 2020-11-12 22:44:52.130484838 +0100 +++ /work/SRC/openSUSE:Factory/.ghc-citeproc.new.5913/ghc-citeproc.changes 2020-11-24 22:14:45.407622172 +0100 @@ -1,0 +2,19 @@ +Thu Nov 19 19:50:28 UTC 2020 - [email protected] + +- Update citeproc to version 0.2. + ## 0.2 + + * Remove `AfterOtherPunctuation` constructor from + `CaseTransformState` [API change]. + This gave bad results with things like parentheses (#27). + * Change `SortKeyValue` to include `Maybe Lang` [API change]. + This allows us to do locale-sensitive sorting (though this + won't matter much unless the `icu` flag is used). + * Add `Maybe Lang` parameter on `initialize` (since + capitalization can be locale-dependent). + * Add cabal.project.icu for building with icu lib. + * Add (unexported) Citeproc.Unicode compatibility module. + This allows us to use the same functions whether or not + the `icu` flag is used. + +------------------------------------------------------------------- Old: ---- citeproc-0.1.1.1.tar.gz New: ---- citeproc-0.2.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ ghc-citeproc.spec ++++++ --- /var/tmp/diff_new_pack.NVWE2x/_old 2020-11-24 22:14:45.959622684 +0100 +++ /var/tmp/diff_new_pack.NVWE2x/_new 2020-11-24 22:14:45.959622684 +0100 @@ -19,7 +19,7 @@ %global pkg_name citeproc %bcond_with tests Name: ghc-%{pkg_name} -Version: 0.1.1.1 +Version: 0.2 Release: 0 Summary: Generates citations and bibliography from CSL styles License: BSD-2-Clause ++++++ citeproc-0.1.1.1.tar.gz -> citeproc-0.2.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.1.1.1/CHANGELOG.md new/citeproc-0.2/CHANGELOG.md --- old/citeproc-0.1.1.1/CHANGELOG.md 2020-11-06 06:09:00.000000000 +0100 +++ new/citeproc-0.2/CHANGELOG.md 2020-11-18 20:34:27.000000000 +0100 @@ -1,5 +1,20 @@ # citeproc changelog +## 0.2 + + * Remove `AfterOtherPunctuation` constructor from + `CaseTransformState` [API change]. + This gave bad results with things like parentheses (#27). + * Change `SortKeyValue` to include `Maybe Lang` [API change]. + This allows us to do locale-sensitive sorting (though this + won't matter much unless the `icu` flag is used). + * Add `Maybe Lang` parameter on `initialize` (since + capitalization can be locale-dependent). + * Add cabal.project.icu for building with icu lib. + * Add (unexported) Citeproc.Unicode compatibility module. + This allows us to use the same functions whether or not + the `icu` flag is used. + ## 0.1.1.1 * Pay attention to citationNoteNumber in computing position. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.1.1.1/app/Main.hs new/citeproc-0.2/app/Main.hs --- old/citeproc-0.1.1.1/app/Main.hs 2020-10-12 06:30:26.000000000 +0200 +++ new/citeproc-0.2/app/Main.hs 2020-11-13 20:03:00.000000000 +0100 @@ -136,7 +136,7 @@ , Option ['r'] ["references"] (ReqArg (\fp opt -> opt{ optReferences = Just fp }) "FILE") "CSL JSON bibliography" - , Option ['a'] ["abbrevations"] + , Option ['a'] ["abbreviations"] (ReqArg (\fp opt -> opt{ optAbbreviations = Just fp }) "FILE") "CSL abbreviations table" , Option ['l'] ["lang"] diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.1.1.1/citeproc.cabal new/citeproc-0.2/citeproc.cabal --- old/citeproc-0.1.1.1/citeproc.cabal 2020-11-06 06:09:07.000000000 +0100 +++ new/citeproc-0.2/citeproc.cabal 2020-11-18 20:30:49.000000000 +0100 @@ -1,6 +1,6 @@ cabal-version: 2.2 name: citeproc -version: 0.1.1.1 +version: 0.2 synopsis: Generates citations and bibliography from CSL styles. description: citeproc parses CSL style files and uses them to generate a list of formatted citations and bibliography @@ -58,6 +58,7 @@ Citeproc.CaseTransform other-modules: Citeproc.Element Citeproc.Data + Citeproc.Unicode build-depends: base >= 4.8 && < 5 , safe diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.1.1.1/man/citeproc.1 new/citeproc-0.2/man/citeproc.1 --- old/citeproc-0.1.1.1/man/citeproc.1 2020-10-23 06:42:15.000000000 +0200 +++ new/citeproc-0.2/man/citeproc.1 2020-11-18 20:34:57.000000000 +0100 @@ -1,6 +1,6 @@ -.\" Automatically generated by Pandoc 2.11.0.2 +.\" Automatically generated by Pandoc 2.11.2 .\" -.TH "citeproc" "1" "" "citeproc 0.1.0.3" "" +.TH "citeproc" "1" "" "citeproc 0.2" "" .hy .SH NAME .PP diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.1.1.1/src/Citeproc/CaseTransform.hs new/citeproc-0.2/src/Citeproc/CaseTransform.hs --- old/citeproc-0.1.1.1/src/Citeproc/CaseTransform.hs 2020-09-27 22:34:12.000000000 +0200 +++ new/citeproc-0.2/src/Citeproc/CaseTransform.hs 2020-11-18 20:22:02.000000000 +0100 @@ -23,6 +23,7 @@ import Data.Text (Text) import qualified Data.Text as T import Citeproc.Types (Lang(..)) +import qualified Citeproc.Unicode as Unicode -- | Wraps a function used to define textcase transformations. newtype CaseTransformer = @@ -36,35 +37,18 @@ | AfterWordEnd | AfterWordChar | AfterSentenceEndingPunctuation - | AfterOtherPunctuation | BeforeLastWord deriving (Show, Eq) -toUpper' :: Maybe Lang -> Text -> Text -toUpper' mblang = T.toUpper . - case mblang of - Just (Lang "tr" _) -> T.map (\c -> case c of - 'i' -> 'İ' - 'ı' -> 'I' - _ -> c) - _ -> id - -toLower' :: Maybe Lang -> Text -> Text -toLower' mblang = T.toLower . - case mblang of - Just (Lang "tr" _) -> T.map (\c -> case c of - 'İ' -> 'i' - 'I' -> 'ı' - _ -> c) - _ -> id - -- | Uppercase everything. withUppercaseAll :: CaseTransformer -withUppercaseAll = CaseTransformer (\mblang _ -> toUpper' mblang) +withUppercaseAll = + CaseTransformer (\mblang _ -> Unicode.toUpper mblang) -- | Lowercase everything. withLowercaseAll :: CaseTransformer -withLowercaseAll = CaseTransformer (\mblang _ -> toLower' mblang) +withLowercaseAll = + CaseTransformer (\mblang _ -> Unicode.toLower mblang) -- | Capitalize all words. withCapitalizeWords :: CaseTransformer @@ -98,10 +82,10 @@ go mblang st chunk | isCapitalized chunk , not (st == Start || st == StartSentence) - = T.toLower chunk + = Unicode.toLower mblang chunk | isCapitalized chunk || T.all isLower chunk , st == Start || st == StartSentence - = capitalizeText mblang $ T.toLower chunk + = capitalizeText mblang $ Unicode.toLower mblang chunk | otherwise = chunk -- | Use title case. @@ -114,12 +98,14 @@ -- textcase_TitleCapitalization.txt | T.any (not . isAscii) chunk = chunk | st == StartSentence || st == Start = - capitalizeText mblang $ T.toLower chunk + capitalizeText mblang $ Unicode.toLower mblang chunk | st == AfterWordEnd , not (isStopWord chunk) - , T.compareLength chunk 1 == GT = capitalizeText mblang $ T.toLower chunk + , T.compareLength chunk 1 == GT = + capitalizeText mblang $ Unicode.toLower mblang chunk | st == BeforeLastWord - , T.compareLength chunk 1 == GT = capitalizeText mblang $ T.toLower chunk + , T.compareLength chunk 1 == GT = + capitalizeText mblang $ Unicode.toLower mblang chunk | otherwise = chunk isCapitalized :: Text -> Bool @@ -134,7 +120,7 @@ capitalizeText :: Maybe Lang -> Text -> Text capitalizeText mblang x = case T.uncons x of - Just (c,x') -> toUpper' mblang (T.singleton c) <> x' + Just (c,x') -> Unicode.toUpper mblang (T.singleton c) <> x' Nothing -> x isStopWord :: Text -> Bool diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.1.1.1/src/Citeproc/CslJson.hs new/citeproc-0.2/src/Citeproc/CslJson.hs --- old/citeproc-0.1.1.1/src/Citeproc/CslJson.hs 2020-10-30 03:18:04.000000000 +0100 +++ new/citeproc-0.2/src/Citeproc/CslJson.hs 2020-11-18 20:22:40.000000000 +0100 @@ -534,7 +534,7 @@ | isSpace c , st == AfterSentenceEndingPunctuation -> StartSentence | isWordBreak c -> AfterWordEnd - | otherwise -> AfterOtherPunctuation + | otherwise -> st return $ if T.all isAlphaNum t then f st t diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.1.1.1/src/Citeproc/Data.hs new/citeproc-0.2/src/Citeproc/Data.hs --- old/citeproc-0.1.1.1/src/Citeproc/Data.hs 2020-09-07 01:10:45.000000000 +0200 +++ new/citeproc-0.2/src/Citeproc/Data.hs 2020-11-09 17:57:04.000000000 +0100 @@ -1,4 +1,3 @@ -{-# LANGUAGE NoImplicitPrelude #-} {-# LANGUAGE TemplateHaskell #-} module Citeproc.Data (localeFiles) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.1.1.1/src/Citeproc/Eval.hs new/citeproc-0.2/src/Citeproc/Eval.hs --- old/citeproc-0.1.1.1/src/Citeproc/Eval.hs 2020-11-06 04:17:42.000000000 +0100 +++ new/citeproc-0.2/src/Citeproc/Eval.hs 2020-11-10 01:54:25.000000000 +0100 @@ -9,6 +9,7 @@ where import Citeproc.Types import Citeproc.Style (mergeLocales) +import qualified Citeproc.Unicode as Unicode import Data.Semigroup import Control.Monad.Trans.RWS.CPS import Data.Containers.ListUtils (nubOrdOn, nubOrd) @@ -400,7 +401,7 @@ , stateNoteMap = mempty })) $ mapM (evalLayout False (styleCitation style)) (zip [1..] (citations ++ ghostCitations)) - + mblang <- asks (localeLanguage . contextLocale) styleOpts <- asks contextStyleOptions let strategy = styleDisambiguation styleOpts let allNameGroups = [ns | Tagged (TagNames _ _ ns) _ <- @@ -423,7 +424,7 @@ , nameFamily v == Just name]) familyNames let toHint names name = - if any (initialsMatch name) (filter (/= name) names) + if any (initialsMatch mblang name) (filter (/= name) names) then case rule of AllNamesWithInitials -> Nothing @@ -463,25 +464,27 @@ case getAmbiguities allCites' of [] -> return $ take (length citations) allCites' ambiguities -> do - analyzeAmbiguities bibSortKeyMap strategy (map snd ambiguities) + analyzeAmbiguities mblang bibSortKeyMap strategy (map snd ambiguities) withRWST (\ctx st -> (ctx, st { stateLastCitedMap = mempty , stateNoteMap = mempty })) $ mapM (evalLayout False (styleCitation style)) (zip [1..] citations) where - analyzeAmbiguities :: M.Map ItemId [SortKeyValue] + analyzeAmbiguities :: Maybe Lang + -> M.Map ItemId [SortKeyValue] -> DisambiguationStrategy -> [[DisambData]] -> Eval a () - analyzeAmbiguities bibSortKeyMap' strategy ambiguities = do + analyzeAmbiguities mblang bibSortKeyMap' strategy ambiguities = do -- add names to et al. as1 <- if disambiguateAddNames strategy - then mapM (tryAddNames (disambiguateAddGivenNames strategy)) + then mapM (tryAddNames mblang + (disambiguateAddGivenNames strategy)) ambiguities else return ambiguities as2 <- case disambiguateAddGivenNames strategy of - Just ByCite -> mapM tryAddGivenNames as1 + Just ByCite -> mapM (tryAddGivenNames mblang) as1 _ -> return as1 as3 <- if disambiguateAddYearSuffix strategy then do @@ -490,12 +493,13 @@ else return as2 mapM_ tryDisambiguateCondition as3 - isDisambiguated :: Maybe GivenNameDisambiguationRule + isDisambiguated :: Maybe Lang + -> Maybe GivenNameDisambiguationRule -> Int -- et al min -> [DisambData] -> DisambData -> Bool - isDisambiguated mbrule etAlMin xs x = + isDisambiguated mblang mbrule etAlMin xs x = all (\y -> x == y || disambiguatedName y /= disambiguatedName x) xs where disambiguatedName = nameParts . take etAlMin . ddNames @@ -503,7 +507,7 @@ case mbrule of Just AllNames -> id Just AllNamesWithInitials -> - map (\name -> name{ nameGiven = initialize True False "" + map (\name -> name{ nameGiven = initialize mblang True False "" <$> nameGiven name }) Just PrimaryName -> \case @@ -512,13 +516,15 @@ Just PrimaryNameWithInitials -> \case [] -> [] - (z:zs) -> z{ nameGiven = initialize True False "" <$> nameGiven z } : + (z:zs) -> z{ nameGiven = + initialize mblang True False "" <$> nameGiven z } : map (\name -> name{ nameGiven = Nothing }) zs Just ByCite -> id -- hints will be added later _ -> map (\name -> name{ nameGiven = Nothing }) - tryAddNames mbrule bs = (case mbrule of - Just ByCite -> bs <$ tryAddGivenNames bs + tryAddNames mblang mbrule bs = + (case mbrule of + Just ByCite -> bs <$ tryAddGivenNames mblang bs _ -> return bs) >>= go 1 -- if ByCite, we want to make sure that -- tryAddGivenNames is still applied, as @@ -528,7 +534,7 @@ go n as | n > maxnames as = return as | otherwise = do - let ds = filter (isDisambiguated mbrule n as) as + let ds = filter (isDisambiguated mblang mbrule n as) as if null ds then go (n + 1) as else do @@ -538,15 +544,16 @@ (unReferenceMap $ stateRefMap st) as } go (n + 1) (as \\ ds) - tryAddGivenNames :: [DisambData] + tryAddGivenNames :: Maybe Lang + -> [DisambData] -> Eval a [DisambData] - tryAddGivenNames as = do + tryAddGivenNames mblang as = do let correspondingNames = map (zip (map ddItem as)) $ transpose $ map ddNames as go [] _ = return [] go (as' :: [DisambData]) (ns :: [(ItemId, Name)]) = do hintedIds <- Set.fromList . catMaybes <$> - mapM (addNameHint (map snd ns)) ns + mapM (addNameHint mblang (map snd ns)) ns return $ filter (\x -> ddItem x `Set.notMember` hintedIds) as' foldM go as correspondingNames @@ -586,20 +593,21 @@ } Just x -> Just x } - initialsMatch x y = + initialsMatch mblang x y = case (nameGiven x, nameGiven y) of (Just x', Just y') -> - initialize True False "" x' == initialize True False "" y' + initialize mblang True False "" x' == + initialize mblang True False "" y' _ -> False - addNameHint names (item, name) = do + addNameHint mblang names (item, name) = do let familyMatches = [n | n <- names , n /= name , nameFamily n == nameFamily name] case familyMatches of [] -> return Nothing _ -> do - let hint = if any (initialsMatch name) familyMatches + let hint = if any (initialsMatch mblang name) familyMatches then AddGivenName else AddInitials modify $ \st -> @@ -842,20 +850,22 @@ -> SortKey a -> Eval a SortKeyValue evalSortKey citeId (SortKeyMacro sortdir elts) = do + mblang <- asks (localeLanguage . contextLocale) refmap <- gets stateRefMap case lookupReference citeId refmap of - Nothing -> return $ SortKeyValue (sortdir, Nothing) + Nothing -> return $ SortKeyValue sortdir mblang Nothing Just ref -> do k <- normalizeSortKey . toText . renderOutput defaultCiteprocOptions . grouped <$> withRWS newContext (mconcat <$> mapM eElement elts) - return $ SortKeyValue (sortdir, Just k) + return $ SortKeyValue sortdir mblang (Just k) where newContext oldContext s = (oldContext, s{ stateReference = ref }) evalSortKey citeId (SortKeyVariable sortdir var) = do + mblang <- asks (localeLanguage . contextLocale) refmap <- gets stateRefMap - SortKeyValue . (sortdir,) <$> + SortKeyValue sortdir mblang <$> case lookupReference citeId refmap >>= lookupVariable var of Nothing -> return Nothing Just (TextVal t) -> return $ Just $ normalizeSortKey t @@ -1981,12 +1991,13 @@ in T.pack [chr (ord 'a' - 1 + (x' `div` 26)), chr (ord 'a' + (x' `mod` 26))] -initialize :: Bool -- ^ initialize +initialize :: Maybe Lang + -> Bool -- ^ initialize -> Bool -- ^ with hyphen -> Text -- ^ initialize with (suffix) -> Text -> Text -initialize makeInitials useHyphen initializeWith = +initialize mblang makeInitials useHyphen initializeWith = T.strip . T.replace " -" "-" . mconcat . map initializeWord . splitWords where -- Left values are already initials @@ -2019,8 +2030,8 @@ case T.uncons t' of Just (c, _) | isUpper c - , useHyphen -> "-" <> T.toUpper (T.singleton c) - | isUpper c -> T.toUpper (T.singleton c) + , useHyphen -> "-" <> Unicode.toUpper mblang (T.singleton c) + | isUpper c -> Unicode.toUpper mblang (T.singleton c) _ -> mempty -- e.g. Ji-ping -> J. not J.-p. Just (c, t') | isUpper c -> @@ -2049,6 +2060,7 @@ asks (styleDemoteNonDroppingParticle . contextStyleOptions) initializeWithHyphen <- asks (styleInitializeWithHyphen . contextStyleOptions) + mblang <- asks (localeLanguage . contextLocale) let initialize' = case nameFamily name of Nothing -> id @@ -2056,6 +2068,7 @@ case nameInitializeWith nameFormat of Just initializeWith -> initialize + mblang (nameInitialize nameFormat) initializeWithHyphen initializeWith diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.1.1.1/src/Citeproc/Pandoc.hs new/citeproc-0.2/src/Citeproc/Pandoc.hs --- old/citeproc-0.1.1.1/src/Citeproc/Pandoc.hs 2020-10-30 03:19:56.000000000 +0100 +++ new/citeproc-0.2/src/Citeproc/Pandoc.hs 2020-11-18 20:22:42.000000000 +0100 @@ -267,7 +267,7 @@ | isSpace c , st == AfterSentenceEndingPunctuation -> StartSentence | isWordBreak c -> AfterWordEnd - | otherwise -> AfterOtherPunctuation + | otherwise -> st return $ if T.all isAlphaNum t then f st t diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.1.1.1/src/Citeproc/Types.hs new/citeproc-0.2/src/Citeproc/Types.hs --- old/citeproc-0.1.1.1/src/Citeproc/Types.hs 2020-10-30 03:18:04.000000000 +0100 +++ new/citeproc-0.2/src/Citeproc/Types.hs 2020-11-10 01:56:43.000000000 +0100 @@ -1,4 +1,3 @@ -{-# LANGUAGE CPP #-} {-# LANGUAGE StrictData #-} {-# LANGUAGE LambdaCase #-} {-# LANGUAGE DeriveFoldable #-} @@ -128,12 +127,8 @@ import qualified Data.Attoparsec.Text as P import Safe (readMay) import Data.String (IsString) - -#ifdef MIN_VERSION_text_icu -import qualified Data.Text.ICU as ICU -#else -import qualified Data.RFC5051 as RFC5051 -#endif +import Citeproc.Unicode (Lang(..), parseLang, renderLang) +import qualified Citeproc.Unicode as Unicode -- import Debug.Trace -- import Text.Show.Pretty (ppShow) @@ -581,43 +576,39 @@ | SortKeyMacro SortDirection [Element a] deriving (Show, Eq) -newtype SortKeyValue = - SortKeyValue (SortDirection, Maybe [Text]) +data SortKeyValue = + SortKeyValue SortDirection (Maybe Lang) (Maybe [Text]) deriving (Show, Eq) -- absence should sort AFTER all values -- see sort_StatusFieldAscending.txt, sort_StatusFieldDescending.txt instance Ord SortKeyValue where - SortKeyValue (Ascending, _) <= SortKeyValue (Ascending, Nothing) + SortKeyValue Ascending _ _ <= SortKeyValue Ascending _ Nothing = True - SortKeyValue (Ascending, Nothing) <= SortKeyValue (Ascending, Just _) + SortKeyValue Ascending _ Nothing <= SortKeyValue Ascending _ (Just _) = False - SortKeyValue (Ascending, Just t1) <= SortKeyValue (Ascending, Just t2) = - t1 `keyLEQ` t2 - SortKeyValue (Descending, _) <= SortKeyValue (Descending, Nothing) + SortKeyValue Ascending mblang (Just t1) <= + SortKeyValue Ascending _ (Just t2) = + keyLEQ mblang t1 t2 + SortKeyValue Descending _ _ <= SortKeyValue Descending _ Nothing = True - SortKeyValue (Descending, Nothing) <= SortKeyValue (Descending, Just _) + SortKeyValue Descending _ Nothing <= SortKeyValue Descending _ (Just _) = False - SortKeyValue (Descending, Just t1) <= SortKeyValue (Descending, Just t2) = - t2 `keyLEQ` t1 - SortKeyValue _ <= SortKeyValue _ = False + SortKeyValue Descending mblang (Just t1) <= + SortKeyValue Descending _ (Just t2) = + keyLEQ mblang t2 t1 + SortKeyValue{} <= SortKeyValue{} = False -- We need special comparison operators to ensure that -- á sorts before b, for example. -keyLEQ :: [Text] -> [Text] -> Bool -keyLEQ _ [] = False -keyLEQ [] _ = True -keyLEQ (x:xs) (y:ys) = - case comp x y of - EQ -> xs `keyLEQ` ys +keyLEQ :: Maybe Lang -> [Text] -> [Text] -> Bool +keyLEQ _ _ [] = False +keyLEQ _ [] _ = True +keyLEQ mblang (x:xs) (y:ys) = + case Unicode.comp mblang x y of + EQ -> keyLEQ mblang xs ys GT -> False LT -> True - where -#ifdef MIN_VERSION_text_icu - comp = ICU.collate (ICU.collator ICU.Current) -#else - comp = RFC5051.compareUnicode -#endif data Layout a = Layout @@ -763,29 +754,6 @@ (isNothing gf1 || isNothing gf2 || gf1 == gf2 ) && (isNothing match1 || isNothing match2 || match1 == match2) --- | A parsed IETF language tag, with language and optional variant. --- For example, @Lang "en" (Just "US")@ corresponds to @en-US@. -data Lang = Lang{ langLanguage :: Text - , langVariant :: Maybe Text } - deriving (Show, Eq, Ord) - -instance ToJSON Lang where - toJSON = toJSON . renderLang - -instance FromJSON Lang where - parseJSON = fmap parseLang . parseJSON - --- | Render a 'Lang' an an IETF language tag. -renderLang :: Lang -> Text -renderLang (Lang l Nothing) = l -renderLang (Lang l (Just v)) = l <> "-" <> v - --- | Parse an IETF language tag. -parseLang :: Text -> Lang -parseLang t = Lang l (snd <$> T.uncons v) - where - (l,v) = T.break (\c -> c == '-' || c == '_') t - -- | Defines locale-specific terms, punctuation styles, and date -- formats. data Locale = diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.1.1.1/src/Citeproc/Unicode.hs new/citeproc-0.2/src/Citeproc/Unicode.hs --- old/citeproc-0.1.1.1/src/Citeproc/Unicode.hs 1970-01-01 01:00:00.000000000 +0100 +++ new/citeproc-0.2/src/Citeproc/Unicode.hs 2020-11-09 18:55:37.000000000 +0100 @@ -0,0 +1,85 @@ +{-# LANGUAGE CPP #-} +{-# LANGUAGE OverloadedStrings #-} +module Citeproc.Unicode + ( Lang(..), + parseLang, + renderLang, + toUpper, + toLower, + comp + ) +where +#ifdef MIN_VERSION_text_icu +import qualified Data.Text.ICU as ICU +#else +import qualified Data.RFC5051 as RFC5051 +#endif +import Data.Text (Text) +import qualified Data.Text as T +import Data.Aeson (FromJSON (..), ToJSON (..)) +import Data.Semigroup + +-- | A parsed IETF language tag, with language and optional variant. +-- For example, @Lang "en" (Just "US")@ corresponds to @en-US@. +data Lang = Lang{ langLanguage :: Text + , langVariant :: Maybe Text } + deriving (Show, Eq, Ord) + +instance ToJSON Lang where + toJSON = toJSON . renderLang + +instance FromJSON Lang where + parseJSON = fmap parseLang . parseJSON + +-- | Render a 'Lang' an an IETF language tag. +renderLang :: Lang -> Text +renderLang (Lang l Nothing) = l +renderLang (Lang l (Just v)) = l <> "-" <> v + +-- | Parse an IETF language tag. +parseLang :: Text -> Lang +parseLang t = Lang l (snd <$> T.uncons v) + where + (l,v) = T.break (\c -> c == '-' || c == '_') t + +#ifdef MIN_VERSION_text_icu +toICULocale :: Maybe Lang -> ICU.LocaleName +toICULocale Nothing = ICU.Current +toICULocale (Just l) = ICU.Locale (T.unpack (renderLang l)) +#endif + +toUpper :: Maybe Lang -> Text -> Text +#ifdef MIN_VERSION_text_icu +toUpper mblang = + ICU.toUpper (toICULocale mblang) +#else +toUpper mblang = T.toUpper . + case mblang of + Just (Lang "tr" _) -> T.map (\c -> case c of + 'i' -> 'İ' + 'ı' -> 'I' + _ -> c) + _ -> id +#endif + +toLower :: Maybe Lang -> Text -> Text +#ifdef MIN_VERSION_text_icu +toLower mblang = + ICU.toLower (toICULocale mblang) +#else +toLower mblang = T.toLower . + case mblang of + Just (Lang "tr" _) -> T.map (\c -> case c of + 'İ' -> 'i' + 'I' -> 'ı' + _ -> c) + _ -> id +#endif + +comp :: Maybe Lang -> Text -> Text -> Ordering +#ifdef MIN_VERSION_text_icu +comp mblang = ICU.collate (ICU.collator (toICULocale mblang)) +#else +comp _mblang = RFC5051.compareUnicode +#endif + diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.1.1.1/test/extra/issue27.txt new/citeproc-0.2/test/extra/issue27.txt --- old/citeproc-0.1.1.1/test/extra/issue27.txt 1970-01-01 01:00:00.000000000 +0100 +++ new/citeproc-0.2/test/extra/issue27.txt 2020-11-18 20:25:54.000000000 +0100 @@ -0,0 +1,690 @@ +>>===== MODE =====>> +bibliography +<<===== MODE =====<< + + + + +>>===== RESULT =====>> +<div class="csl-bib-body"> + <div class="csl-entry">Doe, John. n.d. <i>Fruits (Apples, Pears, Oranges) and Veggies</i>.</div> +</div> +<<===== RESULT =====<< + + +>>===== CSL =====>> + +<?xml version="1.0" encoding="utf-8"?> +<style xmlns="http://purl.org/net/xbiblio/csl"; class="in-text" version="1.0" demote-non-dropping-particle="display-and-sort" page-range-format="chicago"> + <info> + <title>Chicago Manual of Style 17th edition (author-date)</title> + <id>http://www.zotero.org/styles/chicago-author-date</id> + <link href="http://www.zotero.org/styles/chicago-author-date"; rel="self"/> + <link href="http://www.chicagomanualofstyle.org/tools_citationguide.html"; rel="documentation"/> + <author> + <name>Julian Onions</name> + <email>[email protected]</email> + </author> + <contributor> + <name>Sebastian Karcher</name> + </contributor> + <contributor> + <name>Richard Karnesky</name> + <email>[email protected]</email> + <uri>http://arc.nucapt.northwestern.edu/Richard_Karnesky</uri> + </contributor> + <contributor> + <name>Andrew Dunning</name> + <email>[email protected]</email> + <uri>https://orcid.org/0000-0003-0464-5036</uri> + </contributor> + <contributor> + <name>Matthew Roth</name> + <email>[email protected]</email> + <uri> https://orcid.org/0000-0001-7902-6331</uri> + </contributor> + <category citation-format="author-date"/> + <category field="generic-base"/> + <summary>The author-date variant of the Chicago style</summary> + <updated>2018-01-24T12:00:00+00:00</updated> + <rights license="http://creativecommons.org/licenses/by-sa/3.0/";>This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License</rights> + </info> + <locale xml:lang="en"> + <terms> + <term name="editor" form="verb-short">ed.</term> + <term name="container-author" form="verb">by</term> + <term name="translator" form="verb-short">trans.</term> + <term name="editortranslator" form="verb">edited and translated by</term> + <term name="translator" form="short">trans.</term> + </terms> + </locale> + <macro name="secondary-contributors"> + <choose> + <if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="none"> + <group delimiter=". "> + <names variable="editor translator" delimiter=". "> + <label form="verb" text-case="capitalize-first" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + <names variable="director" delimiter=". "> + <label form="verb" text-case="capitalize-first" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + </group> + </if> + </choose> + </macro> + <macro name="container-contributors"> + <choose> + <if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="any"> + <group prefix=", " delimiter=", "> + <names variable="container-author" delimiter=", "> + <label form="verb" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + <names variable="editor translator" delimiter=", "> + <label form="verb" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + </group> + </if> + </choose> + </macro> + <macro name="editor"> + <names variable="editor"> + <name name-as-sort-order="first" and="text" sort-separator=", " delimiter=", " delimiter-precedes-last="always"/> + <label form="short" prefix=", "/> + </names> + </macro> + <macro name="translator"> + <names variable="translator"> + <name name-as-sort-order="first" and="text" sort-separator=", " delimiter=", " delimiter-precedes-last="always"/> + <label form="short" prefix=", "/> + </names> + </macro> + <macro name="recipient"> + <choose> + <if type="personal_communication"> + <choose> + <if variable="genre"> + <text variable="genre" text-case="capitalize-first"/> + </if> + <else> + <text term="letter" text-case="capitalize-first"/> + </else> + </choose> + </if> + </choose> + <names variable="recipient" delimiter=", "> + <label form="verb" prefix=" " text-case="lowercase" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + </macro> + <macro name="substitute-title"> + <choose> + <if type="article-magazine article-newspaper review review-book" match="any"> + <text macro="container-title"/> + </if> + </choose> + </macro> + <macro name="contributors"> + <group delimiter=". "> + <names variable="author"> + <name and="text" name-as-sort-order="first" sort-separator=", " delimiter=", " delimiter-precedes-last="always"/> + <label form="short" prefix=", "/> + <substitute> + <names variable="editor"/> + <names variable="translator"/> + <names variable="director"/> + <text macro="substitute-title"/> + <text macro="title"/> + </substitute> + </names> + <text macro="recipient"/> + </group> + </macro> + <macro name="contributors-short"> + <names variable="author"> + <name form="short" and="text" delimiter=", " initialize-with=". "/> + <substitute> + <names variable="editor"/> + <names variable="translator"/> + <names variable="director"/> + <text macro="substitute-title"/> + <text macro="title"/> + </substitute> + </names> + </macro> + <macro name="interviewer"> + <names variable="interviewer" delimiter=", "> + <label form="verb" prefix=" " text-case="capitalize-first" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + </macro> + <macro name="archive"> + <group delimiter=". "> + <text variable="archive_location" text-case="capitalize-first"/> + <text variable="archive"/> + <text variable="archive-place"/> + </group> + </macro> + <macro name="access"> + <group delimiter=". "> + <choose> + <if type="graphic report" match="any"> + <text macro="archive"/> + </if> + <else-if type="article-journal bill book chapter legal_case legislation motion_picture paper-conference" match="none"> + <text macro="archive"/> + </else-if> + </choose> + <choose> + <if type="webpage post-weblog" match="any"> + <date variable="issued" form="text"/> + </if> + </choose> + <choose> + <if variable="issued" match="none"> + <group delimiter=" "> + <text term="accessed" text-case="capitalize-first"/> + <date variable="accessed" form="text"/> + </group> + </if> + </choose> + <choose> + <if type="legal_case" match="none"> + <choose> + <if variable="DOI"> + <text variable="DOI" prefix="https://doi.org/"/> + </if> + <else> + <text variable="URL"/> + </else> + </choose> + </if> + </choose> + </group> + </macro> + <macro name="title"> + <choose> + <if variable="title" match="none"> + <choose> + <if type="personal_communication" match="none"> + <text variable="genre" text-case="capitalize-first"/> + </if> + </choose> + </if> + <else-if type="bill book graphic legislation motion_picture song" match="any"> + <text variable="title" text-case="title" font-style="italic"/> + <group prefix=" (" suffix=")" delimiter=" "> + <text term="version"/> + <text variable="version"/> + </group> + </else-if> + <else-if variable="reviewed-author"> + <choose> + <if variable="reviewed-title"> + <group delimiter=". "> + <text variable="title" text-case="title" quotes="true"/> + <group delimiter=", "> + <text variable="reviewed-title" text-case="title" font-style="italic" prefix="Review of "/> + <names variable="reviewed-author"> + <label form="verb-short" text-case="lowercase" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + </group> + </group> + </if> + <else> + <group delimiter=", "> + <text variable="title" text-case="title" font-style="italic" prefix="Review of "/> + <names variable="reviewed-author"> + <label form="verb-short" text-case="lowercase" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + </group> + </else> + </choose> + </else-if> + <else-if type="legal_case interview patent" match="any"> + <text variable="title"/> + </else-if> + <else> + <text variable="title" text-case="title" quotes="true"/> + </else> + </choose> + </macro> + <macro name="edition"> + <choose> + <if type="bill book graphic legal_case legislation motion_picture report song" match="any"> + <choose> + <if is-numeric="edition"> + <group delimiter=" " prefix=". "> + <number variable="edition" form="ordinal"/> + <text term="edition" form="short" strip-periods="true"/> + </group> + </if> + <else> + <text variable="edition" text-case="capitalize-first" prefix=". "/> + </else> + </choose> + </if> + <else-if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="any"> + <choose> + <if is-numeric="edition"> + <group delimiter=" " prefix=", "> + <number variable="edition" form="ordinal"/> + <text term="edition" form="short"/> + </group> + </if> + <else> + <text variable="edition" prefix=", "/> + </else> + </choose> + </else-if> + </choose> + </macro> + <macro name="locators"> + <choose> + <if type="article-journal"> + <choose> + <if variable="volume"> + <text variable="volume" prefix=" "/> + <group prefix=" (" suffix=")"> + <choose> + <if variable="issue"> + <text variable="issue"/> + </if> + <else> + <date variable="issued"> + <date-part name="month"/> + </date> + </else> + </choose> + </group> + </if> + <else-if variable="issue"> + <group delimiter=" " prefix=", "> + <text term="issue" form="short"/> + <text variable="issue"/> + <date variable="issued" prefix="(" suffix=")"> + <date-part name="month"/> + </date> + </group> + </else-if> + <else> + <date variable="issued" prefix=", "> + <date-part name="month"/> + </date> + </else> + </choose> + </if> + <else-if type="legal_case"> + <text variable="volume" prefix=", "/> + <text variable="container-title" prefix=" "/> + <text variable="page" prefix=" "/> + </else-if> + <else-if type="bill book graphic legal_case legislation motion_picture report song" match="any"> + <group prefix=". " delimiter=". "> + <group> + <text term="volume" form="short" text-case="capitalize-first" suffix=" "/> + <number variable="volume" form="numeric"/> + </group> + <group> + <number variable="number-of-volumes" form="numeric"/> + <text term="volume" form="short" prefix=" " plural="true"/> + </group> + </group> + </else-if> + <else-if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="any"> + <choose> + <if variable="page" match="none"> + <group prefix=". "> + <text term="volume" form="short" text-case="capitalize-first" suffix=" "/> + <number variable="volume" form="numeric"/> + </group> + </if> + </choose> + </else-if> + </choose> + </macro> + <macro name="locators-chapter"> + <choose> + <if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="any"> + <choose> + <if variable="page"> + <group prefix=", "> + <text variable="volume" suffix=":"/> + <text variable="page"/> + </group> + </if> + </choose> + </if> + </choose> + </macro> + <macro name="locators-article"> + <choose> + <if type="article-newspaper"> + <group prefix=", " delimiter=", "> + <group delimiter=" "> + <text variable="edition"/> + <text term="edition"/> + </group> + <group> + <text term="section" form="short" suffix=" "/> + <text variable="section"/> + </group> + </group> + </if> + <else-if type="article-journal"> + <choose> + <if variable="volume issue" match="any"> + <text variable="page" prefix=": "/> + </if> + <else> + <text variable="page" prefix=", "/> + </else> + </choose> + </else-if> + </choose> + </macro> + <macro name="point-locators"> + <choose> + <if variable="locator"> + <choose> + <if locator="page" match="none"> + <choose> + <if type="bill book graphic legal_case legislation motion_picture report song" match="any"> + <choose> + <if variable="volume"> + <group> + <text term="volume" form="short" suffix=" "/> + <number variable="volume" form="numeric"/> + <label variable="locator" form="short" prefix=", " suffix=" "/> + </group> + </if> + <else> + <label variable="locator" form="short" suffix=" "/> + </else> + </choose> + </if> + <else> + <label variable="locator" form="short" suffix=" "/> + </else> + </choose> + </if> + <else-if type="bill book graphic legal_case legislation motion_picture report song" match="any"> + <number variable="volume" form="numeric" suffix=":"/> + </else-if> + </choose> + <text variable="locator"/> + </if> + </choose> + </macro> + <macro name="container-prefix"> + <text term="in" text-case="capitalize-first"/> + </macro> + <macro name="container-title"> + <choose> + <if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="any"> + <text macro="container-prefix" suffix=" "/> + </if> + </choose> + <choose> + <if type="webpage"> + <text variable="container-title" text-case="title"/> + </if> + <else-if type="legal_case" match="none"> + <group delimiter=" "> + <text variable="container-title" text-case="title" font-style="italic"/> + <choose> + <if type="post-weblog"> + <text value="(blog)"/> + </if> + </choose> + </group> + </else-if> + </choose> + </macro> + <macro name="publisher"> + <group delimiter=": "> + <text variable="publisher-place"/> + <text variable="publisher"/> + </group> + </macro> + <macro name="date"> + <choose> + <if variable="issued"> + <group delimiter=" "> + <date variable="original-date" form="text" date-parts="year" prefix="(" suffix=")"/> + <date variable="issued"> + <date-part name="year"/> + </date> + </group> + </if> + <else-if variable="status"> + <text variable="status" text-case="capitalize-first"/> + </else-if> + <else> + <text term="no date" form="short"/> + </else> + </choose> + </macro> + <macro name="date-in-text"> + <choose> + <if variable="issued"> + <group delimiter=" "> + <date variable="original-date" form="text" date-parts="year" prefix="[" suffix="]"/> + <date variable="issued"> + <date-part name="year"/> + </date> + </group> + </if> + <else-if variable="status"> + <text variable="status"/> + </else-if> + <else> + <text term="no date" form="short"/> + </else> + </choose> + </macro> + <macro name="day-month"> + <date variable="issued"> + <date-part name="month"/> + <date-part name="day" prefix=" "/> + </date> + </macro> + <macro name="collection-title"> + <choose> + <if match="none" type="article-journal"> + <choose> + <if match="none" is-numeric="collection-number"> + <group delimiter=", "> + <text variable="collection-title" text-case="title"/> + <text variable="collection-number"/> + </group> + </if> + <else> + <group delimiter=" "> + <text variable="collection-title" text-case="title"/> + <text variable="collection-number"/> + </group> + </else> + </choose> + </if> + </choose> + </macro> + <macro name="collection-title-journal"> + <choose> + <if type="article-journal"> + <group delimiter=" "> + <text variable="collection-title"/> + <text variable="collection-number"/> + </group> + </if> + </choose> + </macro> + <macro name="event"> + <group> + <text term="presented at" suffix=" "/> + <text variable="event"/> + </group> + </macro> + <macro name="description"> + <choose> + <if type="interview"> + <group delimiter=". "> + <text macro="interviewer"/> + <text variable="medium" text-case="capitalize-first"/> + </group> + </if> + <else-if type="patent"> + <group delimiter=" " prefix=". "> + <text variable="authority"/> + <text variable="number"/> + </group> + </else-if> + <else> + <text variable="medium" text-case="capitalize-first" prefix=". "/> + </else> + </choose> + <choose> + <if variable="title" match="none"/> + <else-if type="thesis personal_communication speech" match="any"/> + <else> + <group delimiter=" " prefix=". "> + <text variable="genre" text-case="capitalize-first"/> + <choose> + <if type="report"> + <text variable="number"/> + </if> + </choose> + </group> + </else> + </choose> + </macro> + <macro name="issue"> + <choose> + <if type="legal_case"> + <text variable="authority" prefix=". "/> + </if> + <else-if type="speech"> + <group prefix=". " delimiter=", "> + <group delimiter=" "> + <text variable="genre" text-case="capitalize-first"/> + <text macro="event"/> + </group> + <text variable="event-place"/> + <text macro="day-month"/> + </group> + </else-if> + <else-if type="article-newspaper article-magazine personal_communication" match="any"> + <date variable="issued" form="text" prefix=", "/> + </else-if> + <else-if type="patent"> + <group delimiter=", " prefix=", "> + <group delimiter=" "> + <!--Needs Localization--> + <text value="filed"/> + <date variable="submitted" form="text"/> + </group> + <group delimiter=" "> + <choose> + <if variable="issued submitted" match="all"> + <text term="and"/> + </if> + </choose> + <!--Needs Localization--> + <text value="issued"/> + <date variable="issued" form="text"/> + </group> + </group> + </else-if> + <else-if type="article-journal" match="any"/> + <else> + <group prefix=". " delimiter=", "> + <choose> + <if type="thesis"> + <text variable="genre" text-case="capitalize-first"/> + </if> + </choose> + <text macro="publisher"/> + </group> + </else> + </choose> + </macro> + <citation et-al-min="4" et-al-use-first="1" disambiguate-add-year-suffix="true" disambiguate-add-names="true" disambiguate-add-givenname="true" givenname-disambiguation-rule="primary-name" collapse="year" after-collapse-delimiter="; "> + <layout prefix="(" suffix=")" delimiter="; "> + <group delimiter=", "> + <choose> + <if variable="issued accessed" match="any"> + <group delimiter=" "> + <text macro="contributors-short"/> + <text macro="date-in-text"/> + </group> + </if> + <!---comma before forthcoming and n.d.--> + <else> + <group delimiter=", "> + <text macro="contributors-short"/> + <text macro="date-in-text"/> + </group> + </else> + </choose> + <text macro="point-locators"/> + </group> + </layout> + </citation> + <bibliography hanging-indent="true" et-al-min="11" et-al-use-first="7" subsequent-author-substitute="———" entry-spacing="0"> + <sort> + <key macro="contributors"/> + <key variable="issued"/> + <key variable="title"/> + </sort> + <layout suffix="."> + <group delimiter=". "> + <text macro="contributors"/> + <text macro="date"/> + <text macro="title"/> + </group> + <text macro="description"/> + <text macro="secondary-contributors" prefix=". "/> + <text macro="container-title" prefix=". "/> + <text macro="container-contributors"/> + <text macro="edition"/> + <text macro="locators-chapter"/> + <text macro="collection-title-journal" prefix=", " suffix=", "/> + <text macro="locators"/> + <text macro="collection-title" prefix=". "/> + <text macro="issue"/> + <text macro="locators-article"/> + <text macro="access" prefix=". "/> + </layout> + </bibliography> +</style> +<<===== CSL =====<< + + + + +>>===== INPUT =====>> +[ + { + "author": [ + { + "family": "Doe", + "given": "John" + } + ], + "id": "doeA", + "title": "Fruits (apples, pears, oranges) and veggies", + "type": "book" + } +] +<<===== INPUT =====<< + + +>>===== VERSION =====>> +1.0 +<<===== VERSION =====<< + _______________________________________________ openSUSE Commits mailing list -- [email protected] To unsubscribe, email [email protected] List Netiquette: https://en.opensuse.org/openSUSE:Mailing_list_netiquette List Archives: https://lists.opensuse.org/archives/list/[email protected]
