Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package ghc-citeproc for openSUSE:Factory checked in at 2025-12-05 16:55:45 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/ghc-citeproc (Old) and /work/SRC/openSUSE:Factory/.ghc-citeproc.new.1939 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "ghc-citeproc" Fri Dec 5 16:55:45 2025 rev:29 rq:1321132 version:0.12 Changes: -------- --- /work/SRC/openSUSE:Factory/ghc-citeproc/ghc-citeproc.changes 2025-10-28 14:48:25.031609004 +0100 +++ /work/SRC/openSUSE:Factory/.ghc-citeproc.new.1939/ghc-citeproc.changes 2025-12-05 16:56:12.807323374 +0100 @@ -1,0 +2,57 @@ +Sun Nov 30 15:56:42 UTC 2025 - Peter Simons <[email protected]> + +- Update citeproc to version 0.12. + ## 0.12 + + * Fix suppression of substituted variables for number, + short- variants (#174). + + * Fix is-numeric detection for non-locators with en-dash (#175). + + * Fix handling of CSL JSON formatting (i.e. handling of + tags like `<i>` and also quotes, which are "smartified") (#170, + #173). + + We were previously handling this in the wrong way, by parsing + it in the `fromText` method for `CslJson Text`. This meant that + there was no way to prevent quotes from turning into smart + quotes, even in things like URLs (see #173). + + What is more, we were not handling this sort of formatting at + all in other contexts, e.g. in terms, `text` elements with a + `value`, locators, prefixes, and suffixes. The CSL tests were + passing only because the raw HTML was being passed through + verbatim, but in other formats this led to undesirable results + (see #170). + + Formatting is now parsed in prefix, suffix, locator, text + value, and terms. And `fromText` is now just `CslText`, so that + purely textual values like URLs can be inserted. + + We also handle the conversion of Unicode superscript characters + to explicit `<sup>.</sup>` in CSL JSON somewhat differently. + Instead of making the replacement in *parsing* CSL JSON, + which would now affect other formats too, because terms in + locales are parsed as CSL JSON, we make the replacement in + *rendering* CSL JSON. This gives us behavior consistent with + the previous behavior, with the substitution occuring in + CSL JSON output but not in Pandoc. + + [API changes] + + + Citeproc.Types now exports `parseCslJson`, `lookupQuotes`, + `superscriptChars`. + + Citeproc.Locale no longer exports `lookupQuotes`. + + `parseCslJson` now takes an additional parameter for a Locale. + + * Add StandardVariable constructor for VariableType [API change] + (#173). Also change the interpretation of StringVariable, + so that StringVariables are not parsed for quotes and HTML-like + formatting. Make URL, ISBN, etc. StringVariables. + + * Test suite improvements: + + + Indicate FAILED:EXPECTED or :UNEXPECTED in output. + + Fix formatting of columns in test summary. + +------------------------------------------------------------------- Old: ---- citeproc-0.11.tar.gz New: ---- citeproc-0.12.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ ghc-citeproc.spec ++++++ --- /var/tmp/diff_new_pack.rC5QCN/_old 2025-12-05 16:56:13.599356488 +0100 +++ /var/tmp/diff_new_pack.rC5QCN/_new 2025-12-05 16:56:13.603356655 +0100 @@ -20,7 +20,7 @@ %global pkgver %{pkg_name}-%{version} %bcond_with tests Name: ghc-%{pkg_name} -Version: 0.11 +Version: 0.12 Release: 0 Summary: Generates citations and bibliography from CSL styles License: BSD-2-Clause ++++++ citeproc-0.11.tar.gz -> citeproc-0.12.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.11/CHANGELOG.md new/citeproc-0.12/CHANGELOG.md --- old/citeproc-0.11/CHANGELOG.md 2001-09-09 03:46:40.000000000 +0200 +++ new/citeproc-0.12/CHANGELOG.md 2001-09-09 03:46:40.000000000 +0200 @@ -1,5 +1,58 @@ # citeproc changelog +## 0.12 + + * Fix suppression of substituted variables for number, + short- variants (#174). + + * Fix is-numeric detection for non-locators with en-dash (#175). + + * Fix handling of CSL JSON formatting (i.e. handling of + tags like `<i>` and also quotes, which are "smartified") (#170, + #173). + + We were previously handling this in the wrong way, by parsing + it in the `fromText` method for `CslJson Text`. This meant that + there was no way to prevent quotes from turning into smart + quotes, even in things like URLs (see #173). + + What is more, we were not handling this sort of formatting at + all in other contexts, e.g. in terms, `text` elements with a + `value`, locators, prefixes, and suffixes. The CSL tests were + passing only because the raw HTML was being passed through + verbatim, but in other formats this led to undesirable results + (see #170). + + Formatting is now parsed in prefix, suffix, locator, text + value, and terms. And `fromText` is now just `CslText`, so that + purely textual values like URLs can be inserted. + + We also handle the conversion of Unicode superscript characters + to explicit `<sup>.</sup>` in CSL JSON somewhat differently. + Instead of making the replacement in *parsing* CSL JSON, + which would now affect other formats too, because terms in + locales are parsed as CSL JSON, we make the replacement in + *rendering* CSL JSON. This gives us behavior consistent with + the previous behavior, with the substitution occuring in + CSL JSON output but not in Pandoc. + + [API changes] + + + Citeproc.Types now exports `parseCslJson`, `lookupQuotes`, + `superscriptChars`. + + Citeproc.Locale no longer exports `lookupQuotes`. + + `parseCslJson` now takes an additional parameter for a Locale. + + * Add StandardVariable constructor for VariableType [API change] + (#173). Also change the interpretation of StringVariable, + so that StringVariables are not parsed for quotes and HTML-like + formatting. Make URL, ISBN, etc. StringVariables. + + * Test suite improvements: + + + Indicate FAILED:EXPECTED or :UNEXPECTED in output. + + Fix formatting of columns in test summary. + ## 0.11 * Expand macros in evaluation rather than style parsing (#172). diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.11/citeproc.cabal new/citeproc-0.12/citeproc.cabal --- old/citeproc-0.11/citeproc.cabal 2001-09-09 03:46:40.000000000 +0200 +++ new/citeproc-0.12/citeproc.cabal 2001-09-09 03:46:40.000000000 +0200 @@ -1,6 +1,6 @@ cabal-version: 2.2 name: citeproc -version: 0.11 +version: 0.12 synopsis: Generates citations and bibliography from CSL styles. description: citeproc parses CSL style files and uses them to generate a list of formatted citations and bibliography diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.11/src/Citeproc/CslJson.hs new/citeproc-0.12/src/Citeproc/CslJson.hs --- old/citeproc-0.11/src/Citeproc/CslJson.hs 2001-09-09 03:46:40.000000000 +0200 +++ new/citeproc-0.12/src/Citeproc/CslJson.hs 2001-09-09 03:46:40.000000000 +0200 @@ -30,19 +30,17 @@ import Citeproc.Types -import Citeproc.Locale (lookupQuotes) import Citeproc.CaseTransform import Data.Ord () -import Data.Char (isAlphaNum, isSpace, isAscii, isPunctuation) +import qualified Data.Map as M +import Data.Char (isAlphaNum, isSpace, isPunctuation) import Data.Text (Text) import qualified Data.Text as T import Data.Foldable (fold) import Data.Functor.Identity -import Data.Attoparsec.Text as P import Data.Aeson (FromJSON(..), ToJSON(..), Value(..), object) import Control.Monad.Trans.State -import Control.Monad (guard, when) -import Control.Applicative ((<|>)) +import Control.Monad (when) import Data.Generics.Uniplate.Direct data CslJson a = @@ -103,7 +101,9 @@ instance CiteprocOutput (CslJson Text) where toText = fold - fromText = parseCslJson mempty + fromText = \t -> if T.null t + then CslEmpty + else CslText t dropTextWhile = dropTextWhile' dropTextWhileEnd = dropTextWhileEnd' addFontVariant x = @@ -183,106 +183,6 @@ CslDiv t x -> CslDiv t (dropTextWhileEnd' f x) CslLink t x -> CslLink t (dropTextWhileEnd' f x) -parseCslJson :: Locale -> Text -> CslJson Text -parseCslJson locale t = - case P.parseOnly - (P.many' (pCslJson locale) <* P.endOfInput) t of - Left _ -> CslText t - Right xs -> mconcat xs - -pCslJson :: Locale -> P.Parser (CslJson Text) -pCslJson locale = P.choice - [ pCslText - , pCslQuoted - , pCslItalic - , pCslBold - , pCslUnderline - , pCslNoDecoration - , pCslSmallCaps - , pCslSup - , pCslSub - , pCslBaseline - , pCslNoCase - , pCslSymbol - ] - where - ((outerOpenQuote, outerCloseQuote), (innerOpenQuote, innerCloseQuote)) = - lookupQuotes locale - isSpecialChar c = c == '<' || c == '>' || c == '\'' || c == '"' || - c == '’' || (not (isAscii c) && (isSuperscriptChar c || isQuoteChar c)) - isQuoteChar = P.inClass - (T.unpack (outerOpenQuote <> outerCloseQuote <> - innerOpenQuote <> innerCloseQuote)) - isSuperscriptChar = P.inClass superscriptChars - isApostrophe '\'' = True - isApostrophe '’' = True - isApostrophe _ = False - pCsl = pCslJson locale - notFollowedBySpace = - P.peekChar' >>= guard . not . isSpaceChar - isSpaceChar = P.inClass [' ','\t','\n','\r'] - pOpenQuote = (("\"" <$ P.char '"') - <|> ("'" <$ P.char '\'') - <|> (outerCloseQuote <$ P.string outerOpenQuote) - <|> (innerCloseQuote <$ P.string innerOpenQuote)) - <* notFollowedBySpace - pSpace = P.skipWhile isSpaceChar - pCslText = CslText . addNarrowSpace <$> - ( do t <- P.takeWhile1 (\c -> isAlphaNum c && not (isSpecialChar c)) - -- apostrophe - P.option t $ do _ <- P.satisfy isApostrophe - t' <- P.takeWhile1 isAlphaNum - return (t <> "’" <> t') - <|> - (P.takeWhile1 (\c -> not (isAlphaNum c || isSpecialChar c))) ) - pCslQuoted = CslQuoted <$> - do cl <- pOpenQuote - mbc <- peekChar - case mbc of - Just c | T.singleton c == cl -> fail "unexpected close quote" - _ -> return () - mconcat <$> P.manyTill' pCsl (P.string cl) - pCslSymbol = do - c <- P.satisfy isSpecialChar - return $ - if isApostrophe c - then CslText "’" - else charToSup c - pCslItalic = CslItalic . mconcat <$> - (P.string "<i>" *> P.manyTill' pCsl (P.string "</i>")) - pCslBold = CslBold . mconcat <$> - (P.string "<b>" *> P.manyTill' pCsl (P.string "</b>")) - pCslUnderline = CslUnderline . mconcat <$> - (P.string "<u>" *> P.manyTill' pCsl (P.string "</u>")) - pCslNoDecoration = CslNoDecoration . mconcat <$> - (P.string "<span" *> pSpace *> - P.string "class=\"nodecor\"" *> pSpace *> P.char '>' *> - P.manyTill' pCsl (P.string "</span>")) - pCslSup = CslSup . mconcat <$> - (P.string "<sup>" *> P.manyTill' pCsl (P.string "</sup>")) - pCslSub = CslSub . mconcat <$> - (P.string "<sub>" *> P.manyTill' pCsl (P.string "</sub>")) - pCslBaseline = CslBaseline . mconcat <$> - (P.string "<span" *> pSpace *> P.string "style=\"baseline\">" *> - P.manyTill' pCsl (P.string "</span>")) - pCslSmallCaps = CslSmallCaps . mconcat <$> - ((P.string "<span" *> pSpace *> - P.string "style=\"font-variant:" *> pSpace *> - P.string "small-caps;" *> pSpace *> P.char '"' *> - pSpace *> P.char '>' *> P.manyTill' pCsl (P.string "</span>")) - <|> - (P.string "<sc>" *> P.manyTill' pCsl (P.string "</sc>"))) - pCslNoCase = CslNoCase . mconcat <$> - (P.string "<span" *> pSpace *> - P.string "class=\"nocase\"" *> pSpace *> P.char '>' *> - P.manyTill' pCsl (P.string "</span>")) - addNarrowSpace = - T.replace " ;" "\x202F;" . - T.replace " ?" "\x202F?" . - T.replace " !" "\x202F!" . - T.replace " »" "\x202F»" . - T.replace "« " "«\x202F" - data RenderContext = RenderContext { useOuterQuotes :: Bool @@ -305,7 +205,11 @@ go :: RenderContext -> CslJson Text -> Text go ctx el = case el of - CslText t -> escape t + CslText t + | Just (c, "") <- T.uncons t + , Just t' <- M.lookup c superscriptChars + -> "<sup>" <> t' <> "</sup>" + | otherwise -> escape t CslEmpty -> mempty CslConcat x y -> go ctx x <> go ctx y CslQuoted x @@ -611,219 +515,3 @@ CslText t -> CslText t CslEmpty -> CslEmpty -superscriptChars :: [Char] -superscriptChars = - [ '\x00AA' - , '\x00B2' - , '\x00B3' - , '\x00B9' - , '\x00BA' - , '\x02B0' - , '\x02B1' - , '\x02B2' - , '\x02B3' - , '\x02B4' - , '\x02B5' - , '\x02B6' - , '\x02B7' - , '\x02B8' - , '\x02E0' - , '\x02E1' - , '\x02E2' - , '\x02E3' - , '\x02E4' - , '\x1D2C' - , '\x1D2D' - , '\x1D2E' - , '\x1D30' - , '\x1D31' - , '\x1D32' - , '\x1D33' - , '\x1D34' - , '\x1D35' - , '\x1D36' - , '\x1D37' - , '\x1D38' - , '\x1D39' - , '\x1D3A' - , '\x1D3C' - , '\x1D3D' - , '\x1D3E' - , '\x1D3F' - , '\x1D40' - , '\x1D41' - , '\x1D42' - , '\x1D43' - , '\x1D44' - , '\x1D45' - , '\x1D46' - , '\x1D47' - , '\x1D48' - , '\x1D49' - , '\x1D4A' - , '\x1D4B' - , '\x1D4C' - , '\x1D4D' - , '\x1D4F' - , '\x1D50' - , '\x1D51' - , '\x1D52' - , '\x1D53' - , '\x1D54' - , '\x1D55' - , '\x1D56' - , '\x1D57' - , '\x1D58' - , '\x1D59' - , '\x1D5A' - , '\x1D5B' - , '\x1D5C' - , '\x1D5D' - , '\x1D5E' - , '\x1D5F' - , '\x1D60' - , '\x1D61' - , '\x2070' - , '\x2071' - , '\x2074' - , '\x2075' - , '\x2076' - , '\x2077' - , '\x2078' - , '\x2079' - , '\x207A' - , '\x207B' - , '\x207C' - , '\x207D' - , '\x207E' - , '\x207F' - , '\x2120' - , '\x2122' - , '\x3192' - , '\x3193' - , '\x3194' - , '\x3195' - , '\x3196' - , '\x3197' - , '\x3198' - , '\x3199' - , '\x319A' - , '\x319B' - , '\x319C' - , '\x319D' - , '\x319E' - , '\x319F' - , '\x02C0' - , '\x02C1' - , '\x06E5' - , '\x06E6' - ] - -charToSup :: Char -> CslJson Text -charToSup c = - case c of - '\x00AA' -> CslSup (CslText "\x0061") - '\x00B2' -> CslSup (CslText "\x0032") - '\x00B3' -> CslSup (CslText "\x0033") - '\x00B9' -> CslSup (CslText "\x0031") - '\x00BA' -> CslSup (CslText "\x006F") - '\x02B0' -> CslSup (CslText "\x0068") - '\x02B1' -> CslSup (CslText "\x0266") - '\x02B2' -> CslSup (CslText "\x006A") - '\x02B3' -> CslSup (CslText "\x0072") - '\x02B4' -> CslSup (CslText "\x0279") - '\x02B5' -> CslSup (CslText "\x027B") - '\x02B6' -> CslSup (CslText "\x0281") - '\x02B7' -> CslSup (CslText "\x0077") - '\x02B8' -> CslSup (CslText "\x0079") - '\x02E0' -> CslSup (CslText "\x0263") - '\x02E1' -> CslSup (CslText "\x006C") - '\x02E2' -> CslSup (CslText "\x0073") - '\x02E3' -> CslSup (CslText "\x0078") - '\x02E4' -> CslSup (CslText "\x0295") - '\x1D2C' -> CslSup (CslText "\x0041") - '\x1D2D' -> CslSup (CslText "\x00C6") - '\x1D2E' -> CslSup (CslText "\x0042") - '\x1D30' -> CslSup (CslText "\x0044") - '\x1D31' -> CslSup (CslText "\x0045") - '\x1D32' -> CslSup (CslText "\x018E") - '\x1D33' -> CslSup (CslText "\x0047") - '\x1D34' -> CslSup (CslText "\x0048") - '\x1D35' -> CslSup (CslText "\x0049") - '\x1D36' -> CslSup (CslText "\x004A") - '\x1D37' -> CslSup (CslText "\x004B") - '\x1D38' -> CslSup (CslText "\x004C") - '\x1D39' -> CslSup (CslText "\x004D") - '\x1D3A' -> CslSup (CslText "\x004E") - '\x1D3C' -> CslSup (CslText "\x004F") - '\x1D3D' -> CslSup (CslText "\x0222") - '\x1D3E' -> CslSup (CslText "\x0050") - '\x1D3F' -> CslSup (CslText "\x0052") - '\x1D40' -> CslSup (CslText "\x0054") - '\x1D41' -> CslSup (CslText "\x0055") - '\x1D42' -> CslSup (CslText "\x0057") - '\x1D43' -> CslSup (CslText "\x0061") - '\x1D44' -> CslSup (CslText "\x0250") - '\x1D45' -> CslSup (CslText "\x0251") - '\x1D46' -> CslSup (CslText "\x1D02") - '\x1D47' -> CslSup (CslText "\x0062") - '\x1D48' -> CslSup (CslText "\x0064") - '\x1D49' -> CslSup (CslText "\x0065") - '\x1D4A' -> CslSup (CslText "\x0259") - '\x1D4B' -> CslSup (CslText "\x025B") - '\x1D4C' -> CslSup (CslText "\x025C") - '\x1D4D' -> CslSup (CslText "\x0067") - '\x1D4F' -> CslSup (CslText "\x006B") - '\x1D50' -> CslSup (CslText "\x006D") - '\x1D51' -> CslSup (CslText "\x014B") - '\x1D52' -> CslSup (CslText "\x006F") - '\x1D53' -> CslSup (CslText "\x0254") - '\x1D54' -> CslSup (CslText "\x1D16") - '\x1D55' -> CslSup (CslText "\x1D17") - '\x1D56' -> CslSup (CslText "\x0070") - '\x1D57' -> CslSup (CslText "\x0074") - '\x1D58' -> CslSup (CslText "\x0075") - '\x1D59' -> CslSup (CslText "\x1D1D") - '\x1D5A' -> CslSup (CslText "\x026F") - '\x1D5B' -> CslSup (CslText "\x0076") - '\x1D5C' -> CslSup (CslText "\x1D25") - '\x1D5D' -> CslSup (CslText "\x03B2") - '\x1D5E' -> CslSup (CslText "\x03B3") - '\x1D5F' -> CslSup (CslText "\x03B4") - '\x1D60' -> CslSup (CslText "\x03C6") - '\x1D61' -> CslSup (CslText "\x03C7") - '\x2070' -> CslSup (CslText "\x0030") - '\x2071' -> CslSup (CslText "\x0069") - '\x2074' -> CslSup (CslText "\x0034") - '\x2075' -> CslSup (CslText "\x0035") - '\x2076' -> CslSup (CslText "\x0036") - '\x2077' -> CslSup (CslText "\x0037") - '\x2078' -> CslSup (CslText "\x0038") - '\x2079' -> CslSup (CslText "\x0039") - '\x207A' -> CslSup (CslText "\x002B") - '\x207B' -> CslSup (CslText "\x2212") - '\x207C' -> CslSup (CslText "\x003D") - '\x207D' -> CslSup (CslText "\x0028") - '\x207E' -> CslSup (CslText "\x0029") - '\x207F' -> CslSup (CslText "\x006E") - '\x2120' -> CslSup (CslText "\x0053\x004D") - '\x2122' -> CslSup (CslText "\x0054\x004D") - '\x3192' -> CslSup (CslText "\x4E00") - '\x3193' -> CslSup (CslText "\x4E8C") - '\x3194' -> CslSup (CslText "\x4E09") - '\x3195' -> CslSup (CslText "\x56DB") - '\x3196' -> CslSup (CslText "\x4E0A") - '\x3197' -> CslSup (CslText "\x4E2D") - '\x3198' -> CslSup (CslText "\x4E0B") - '\x3199' -> CslSup (CslText "\x7532") - '\x319A' -> CslSup (CslText "\x4E59") - '\x319B' -> CslSup (CslText "\x4E19") - '\x319C' -> CslSup (CslText "\x4E01") - '\x319D' -> CslSup (CslText "\x5929") - '\x319E' -> CslSup (CslText "\x5730") - '\x319F' -> CslSup (CslText "\x4EBA") - '\x02C0' -> CslSup (CslText "\x0294") - '\x02C1' -> CslSup (CslText "\x0295") - '\x06E5' -> CslSup (CslText "\x0648") - '\x06E6' -> CslSup (CslText "\x064A") - _ -> CslText $ T.singleton c diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.11/src/Citeproc/Eval.hs new/citeproc-0.12/src/Citeproc/Eval.hs --- old/citeproc-0.11/src/Citeproc/Eval.hs 2001-09-09 03:46:40.000000000 +0200 +++ new/citeproc-0.12/src/Citeproc/Eval.hs 2001-09-09 03:46:40.000000000 +0200 @@ -1024,8 +1024,9 @@ case lookupReference citeId refmap of Nothing -> return $ SortKeyValue sortdir Nothing Just ref -> do + locale <- asks contextLocale k <- normalizeSortKey . toText . - renderOutput defaultCiteprocOptions . grouped + renderOutput defaultCiteprocOptions locale . grouped <$> withRWS newContext (eElements elts) return $ SortKeyValue sortdir (Just k) where @@ -1489,34 +1490,34 @@ Verb -> lookupTerm' term{ termForm = Long } Short -> lookupTerm' term{ termForm = Long } _ -> return NullOutput - f ((_,t):_) = return $ if T.null t - then NullOutput - else Literal $ fromText t + f ((_,t):_) = if T.null t + then return NullOutput + else do + locale <- asks contextLocale + return $ Literal $ parseCslJson locale t pageRange :: CiteprocOutput a => Text -> Eval a (Output a) pageRange x = do - pageDelim <- lookupTerm' - emptyTerm{ termName = "page-range-delimiter" } + delimTerms <- lookupTerm emptyTerm{ termName = "page-range-delimiter" } + let pageDelim = case delimTerms of + (_,d):_ -> d + _ -> "" mbPageRangeFormat <- asks (stylePageRangeFormat . contextStyleOptions) let ranges = map T.strip $ T.groupBy (\c d -> not (c == ',' || c == '&' || d == ',' || d == '&')) x return $ formatted mempty{ formatDelimiter = Just " " } - $ map (formatPageRange mbPageRangeFormat - (case pageDelim of - NullOutput -> literal $ T.singleton enDash - delim -> delim)) ranges + $ map (literal . formatPageRange mbPageRangeFormat pageDelim) ranges enDash :: Char enDash = '\x2013' -formatPageRange :: CiteprocOutput a - => Maybe PageRangeFormat - -> Output a +formatPageRange :: Maybe PageRangeFormat -> Text - -> Output a -formatPageRange _ _ "&" = literal "&" -formatPageRange _ _ "," = literal "," + -> Text + -> Text +formatPageRange _ _ "&" = "&" +formatPageRange _ _ "," = "," formatPageRange mbPageRangeFormat delim t = let isDash '-' = True isDash '\x2013' = True @@ -1525,9 +1526,9 @@ then [T.replace "\\-" "-" t] else map T.strip $ T.split isDash t inRange pref xs - | T.null pref = grouped (intersperse delim (map literal xs)) - | otherwise = grouped - (literal pref : intersperse delim (map literal xs)) + | T.null pref = mconcat (intersperse delim xs) + | otherwise = mconcat + (pref : intersperse delim xs) changedDigits xs ys = length $ filter not $ zipWith (==) (xs ++ repeat ' ') ys minimal threshold pref x y = @@ -1538,8 +1539,8 @@ else inRange pref [x, resty] _ -> inRange pref [x, y] in case rangeParts of - [] -> NullOutput - [w] -> literal w + [] -> "" + [w] -> w [w,v] | Nothing <- mbPageRangeFormat -> inRange mempty [w,v] | Just fmt <- mbPageRangeFormat -> do @@ -1578,7 +1579,7 @@ PageRangeMinimal -> minimal 1 pref x y' PageRangeMinimalTwo -> minimal 2 pref x y' else inRange mempty [w,v] - _ -> literal t + _ -> t eText :: CiteprocOutput a => TextType -> Eval a (Output a) eText (TextVariable varForm v) = do @@ -1611,9 +1612,10 @@ Tagged TagLocator <$> pageRange x | otherwise -> do updateVarCount 1 1 + locale <- asks contextLocale return $ Tagged TagLocator $ - formatPageRange Nothing - (literal $ T.singleton enDash) x + Literal . parseCslJson locale $ + formatPageRange Nothing (T.singleton enDash) x Nothing -> NullOutput <$ updateVarCount 1 0 "year-suffix" -> do @@ -1683,7 +1685,10 @@ Just (NumVal x) -> return $ Literal $ fromText (T.pack (show x)) _ -> return NullOutput - unless (isNothing mbv) $ deleteSubstitutedVariables [v] + unless (isNothing mbv) $ deleteSubstitutedVariables + (v : if varForm == ShortForm -- see #174 + then [v <> "-short"] + else []) if v == "title" && res /= NullOutput then do modify (\st -> st { stateUsedTitle = True }) @@ -1705,7 +1710,9 @@ eText (TextMacro name) = do warn ("unexpanded macro " <> name) return NullOutput -eText (TextValue t) = return $ Literal $ fromText t +eText (TextValue t) = do + locale <- asks contextLocale + return . Literal $ parseCslJson locale t eText (TextTerm term) = do t' <- lookupTerm' term t'' <- if termName term == "no date" @@ -2635,7 +2642,7 @@ | T.all isDigit t' -> True | otherwise -> False let isNumeric t = all isNumericChunk - (T.split (\c -> c == ',' || c == '-' || c == '&') t) + (T.split (\c -> c == ',' || c == '-' || c == '&' || c == '\x2013') t) let testCondition cond = case cond of HasVariable "locator" -> hasLocator @@ -2684,6 +2691,7 @@ Just (FancyVal x) -> splitNums (toText x) Just (TextVal t) -> splitNums t _ -> [] + unless (isNothing mbv) $ deleteSubstitutedVariables [var] grouped <$> mapM (evalNumber nform mbGender) nparts evalNumber :: CiteprocOutput a @@ -2701,9 +2709,10 @@ res <- (if i > 99 then filter (\(t,_) -> termMatch t /= Just WholeNumber) else id) <$> lookupTerm twomatch + locale <- asks contextLocale case res of ((_,suff):_) -> - return $ Literal $ fromText (dectext <> suff) + return $ Literal $ fromText dectext <> parseCslJson locale suff [] -> do -- not an exact match res' <- (if i > 10 then filter (\(t,_) -> @@ -2712,12 +2721,12 @@ else id) <$> lookupTerm onematch case res' of ((_,suff):_) -> - return $ Literal $ fromText (dectext <> suff) + return $ Literal $ fromText dectext <> parseCslJson locale suff [] -> do res'' <- lookupTerm fallback case res'' of ((_,suff):_) -> - return $ Literal $ fromText (dectext <> suff) + return $ Literal $ fromText dectext <> parseCslJson locale suff [] -> do warn $ "no ordinal suffix found for " <> dectext return $ Literal $ fromText (T.pack (show i)) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.11/src/Citeproc/Locale.hs new/citeproc-0.12/src/Citeproc/Locale.hs --- old/citeproc-0.11/src/Citeproc/Locale.hs 2001-09-09 03:46:40.000000000 +0200 +++ new/citeproc-0.12/src/Citeproc/Locale.hs 2001-09-09 03:46:40.000000000 +0200 @@ -3,15 +3,13 @@ module Citeproc.Locale ( parseLocale, getLocale, - getPrimaryDialect, - lookupQuotes + getPrimaryDialect ) where import Citeproc.Types import Citeproc.Element (runElementParser, pLocale) import Citeproc.Data (localeFiles) import qualified Data.Map as M -import Data.Maybe (fromMaybe) import Data.Text (Text) import qualified Text.XML as X import System.FilePath (takeExtension, dropExtension) @@ -108,18 +106,3 @@ Just loc -> loc Nothing -> Left $ CiteprocLocaleNotFound $ renderLang lang -lookupTerm :: Locale -> Text -> Maybe Text -lookupTerm locale termname = do - let terms = localeTerms locale - case M.lookup termname terms of - Just ((_,t):_) -> Just t - _ -> Nothing - -lookupQuotes :: Locale -> ((Text, Text), (Text, Text)) -lookupQuotes locale = ((outerOpen, outerClose), (innerOpen, innerClose)) - where - outerOpen = fromMaybe "\x201C" $ lookupTerm locale "open-quote" - outerClose = fromMaybe "\x201D" $ lookupTerm locale "close-quote" - innerOpen = fromMaybe "\x2018" $ lookupTerm locale "open-inner-quote" - innerClose = fromMaybe "\x2019" $ lookupTerm locale "close-inner-quote" - diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.11/src/Citeproc/Pandoc.hs new/citeproc-0.12/src/Citeproc/Pandoc.hs --- old/citeproc-0.11/src/Citeproc/Pandoc.hs 2001-09-09 03:46:40.000000000 +0200 +++ new/citeproc-0.12/src/Citeproc/Pandoc.hs 2001-09-09 03:46:40.000000000 +0200 @@ -17,7 +17,6 @@ import Citeproc.CaseTransform import Control.Monad.Trans.State.Strict as S import Control.Monad (unless, when) -import Citeproc.Locale (lookupQuotes) import Data.Functor.Reverse import Data.Char (isSpace, isPunctuation, isAlphaNum) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.11/src/Citeproc/Types.hs new/citeproc-0.12/src/Citeproc/Types.hs --- old/citeproc-0.11/src/Citeproc/Types.hs 2001-09-09 03:46:40.000000000 +0200 +++ new/citeproc-0.12/src/Citeproc/Types.hs 2001-09-09 03:46:40.000000000 +0200 @@ -107,6 +107,9 @@ , lookupAbbreviation , Result(..) , Inputs(..) + , parseCslJson + , lookupQuotes + , superscriptChars ) where import qualified Data.Set as Set @@ -116,7 +119,7 @@ import qualified Data.CaseInsensitive as CI import Control.Monad (foldM, guard, mzero) import Control.Applicative ((<|>), optional) -import Data.Char (isLower, isDigit, isLetter, isSpace) +import Data.Char (isLower, isDigit, isLetter, isSpace, isAlphaNum, isAscii) import Data.Text (Text) import GHC.Generics (Generic) import qualified Data.Text as T @@ -212,8 +215,8 @@ addHyperlink :: Text -> a -> a localizeQuotes :: Locale -> a -> a -addFormatting :: CiteprocOutput a => Formatting -> a -> a -addFormatting f x = +addFormatting :: CiteprocOutput a => Locale -> Formatting -> a -> a +addFormatting locale f x = if T.null (toText x) -- TODO inefficient then mempty else @@ -231,10 +234,10 @@ $ x where addPrefix z = case formatPrefix f of - Just s -> mconcat $ fixPunct [fromText s, z] + Just s -> mconcat $ fixPunct [parseCslJson locale s, z] Nothing -> z addSuffix z = case formatSuffix f of - Just s -> mconcat $ fixPunct [z, fromText s] + Just s -> mconcat $ fixPunct [z, parseCslJson locale s] Nothing -> z affixesInside = formatAffixesInside f @@ -931,6 +934,9 @@ | otherwise = Reference i t d <$> case variableType k of StringVariable -> do + v' <- TextVal <$> readString v + return $ M.insert k v' m + StandardVariable -> do v' <- FancyVal <$> parseJSON v <|> TextVal <$> readString v return $ M.insert k v' m NumberVariable -> do @@ -946,7 +952,7 @@ NameVariable -> do v' <- parseJSON v return $ M.insert k (NamesVal v') m - UnknownVariable -> -- treat as string variable if possible + UnknownVariable -> -- treat as standard variable if possible case v of String{} -> (\x -> M.insert k x m) <$> (FancyVal <$> parseJSON v <|> TextVal <$> readString v) @@ -996,6 +1002,7 @@ | NameVariable | NumberVariable | StringVariable + | StandardVariable | UnknownVariable deriving (Show, Eq) @@ -1052,53 +1059,53 @@ variableType "supplement-number" = NumberVariable variableType "version" = NumberVariable variableType "volume" = NumberVariable -variableType "abstract" = StringVariable -variableType "annote" = StringVariable -variableType "archive" = StringVariable -variableType "archive_collection" = StringVariable -variableType "archive_location" = StringVariable -variableType "archive-place" = StringVariable -variableType "authority" = StringVariable -variableType "call-number" = StringVariable -variableType "citation-key" = StringVariable -variableType "citation-label" = StringVariable -variableType "collection-title" = StringVariable -variableType "container-title" = StringVariable -variableType "container-title-short" = StringVariable -variableType "dimensions" = StringVariable -variableType "division" = StringVariable +variableType "abstract" = StandardVariable +variableType "annote" = StandardVariable +variableType "archive" = StandardVariable +variableType "archive_collection" = StandardVariable +variableType "archive_location" = StandardVariable +variableType "archive-place" = StandardVariable +variableType "authority" = StandardVariable +variableType "call-number" = StandardVariable +variableType "citation-key" = StandardVariable +variableType "citation-label" = StandardVariable +variableType "collection-title" = StandardVariable +variableType "container-title" = StandardVariable +variableType "container-title-short" = StandardVariable +variableType "dimensions" = StandardVariable +variableType "division" = StandardVariable variableType "DOI" = StringVariable -variableType "event" = StringVariable -variableType "event-place" = StringVariable -variableType "event-title" = StringVariable --(new name for "event" to avoid confusion with new "event" type) -variableType "genre" = StringVariable +variableType "event" = StandardVariable +variableType "event-place" = StandardVariable +variableType "event-title" = StandardVariable --(new name for "event" to avoid confusion with new "event" type) +variableType "genre" = StandardVariable variableType "ISBN" = StringVariable variableType "ISSN" = StringVariable -variableType "jurisdiction" = StringVariable -variableType "keyword" = StringVariable -variableType "language" = StringVariable -variableType "license" = StringVariable -variableType "medium" = StringVariable -variableType "note" = StringVariable -variableType "original-publisher" = StringVariable -variableType "original-publisher-place" = StringVariable -variableType "original-title" = StringVariable -variableType "part-title" = StringVariable +variableType "jurisdiction" = StandardVariable +variableType "keyword" = StandardVariable +variableType "language" = StandardVariable +variableType "license" = StandardVariable +variableType "medium" = StandardVariable +variableType "note" = StandardVariable +variableType "original-publisher" = StandardVariable +variableType "original-publisher-place" = StandardVariable +variableType "original-title" = StandardVariable +variableType "part-title" = StandardVariable variableType "PMID" = StringVariable variableType "PMCID" = StringVariable -variableType "publisher" = StringVariable -variableType "publisher-place" = StringVariable -variableType "references" = StringVariable -variableType "reviewed-genre" = StringVariable -variableType "reviewed-title" = StringVariable -variableType "scale" = StringVariable -variableType "source" = StringVariable -variableType "status" = StringVariable -variableType "title" = StringVariable -variableType "title-short" = StringVariable +variableType "publisher" = StandardVariable +variableType "publisher-place" = StandardVariable +variableType "references" = StandardVariable +variableType "reviewed-genre" = StandardVariable +variableType "reviewed-title" = StandardVariable +variableType "scale" = StandardVariable +variableType "source" = StandardVariable +variableType "status" = StandardVariable +variableType "title" = StandardVariable +variableType "title-short" = StandardVariable variableType "URL" = StringVariable -variableType "volume-title" = StringVariable -variableType "year-suffix" = StringVariable +variableType "volume-title" = StandardVariable +variableType "year-suffix" = StandardVariable variableType _ = UnknownVariable newtype (ReferenceMap a) = @@ -1571,42 +1578,42 @@ outputToText (Linked _ xs) = T.unwords $ map outputToText xs outputToText (InNote x) = outputToText x -renderOutput :: CiteprocOutput a => CiteprocOptions -> Output a -> a -renderOutput _ NullOutput = mempty -renderOutput _ (Literal x) = x -renderOutput opts (Tagged (TagItem itemtype ident) x) +renderOutput :: CiteprocOutput a => CiteprocOptions -> Locale -> Output a -> a +renderOutput _ _ NullOutput = mempty +renderOutput _ _ (Literal x) = x +renderOutput opts locale (Tagged (TagItem itemtype ident) x) | linkCitations opts , itemtype /= AuthorOnly - = addHyperlink ("#ref-" <> unItemId ident) $ renderOutput opts x -renderOutput opts (Tagged (TagNames _ _ ns) x) + = addHyperlink ("#ref-" <> unItemId ident) $ renderOutput opts locale x +renderOutput opts locale (Tagged (TagNames _ _ ns) x) -- a hack to ensure that names starting with lowercase don't get -- capitalized in pandoc footnotes: see jgm/pandoc#10983 | any hasNonstandardCase ns - = addTextCase Nothing PreserveCase $ renderOutput opts x -renderOutput opts (Tagged _ x) = renderOutput opts x -renderOutput opts (Formatted f [Linked url xs]) + = addTextCase Nothing PreserveCase $ renderOutput opts locale x +renderOutput opts locale (Tagged _ x) = renderOutput opts locale x +renderOutput opts locale (Formatted f [Linked url xs]) | linkBibliography opts , url == prefix <> anchor -- ensure correct handling of link prefixes like (https://doi.org/) -- when a link's prefix+anchor=target, ensure the link includes the prefix -- (see pandoc#6723 and citeproc#88) - = renderOutput opts $ Linked url [Formatted f xs] + = renderOutput opts locale $ Linked url [Formatted f xs] where anchor = mconcat (map outputToText xs) prefix = fromMaybe "" (formatPrefix f) -renderOutput opts (Formatted formatting xs) = - addFormatting formatting . mconcat . fixPunct . +renderOutput opts locale (Formatted formatting xs) = + addFormatting locale formatting . mconcat . fixPunct . (case formatDelimiter formatting of Just d -> addDelimiters (fromText d) - Nothing -> id) . filter (/= mempty) $ map (renderOutput opts) xs -renderOutput opts (Linked url xs) + Nothing -> id) . filter (/= mempty) $ map (renderOutput opts locale) xs +renderOutput opts locale (Linked url xs) = (if linkBibliography opts then addHyperlink url - else id) . mconcat . fixPunct $ map (renderOutput opts) xs -renderOutput opts (InNote x) = inNote $ + else id) . mconcat . fixPunct $ map (renderOutput opts locale) xs +renderOutput opts locale (InNote x) = inNote $ dropTextWhile isSpace $ dropTextWhile (\c -> c == ',' || c == ';' || c == '.' || c == ':') $ - renderOutput opts x + renderOutput opts locale x hasNonstandardCase :: Name -> Bool hasNonstandardCase name = @@ -1810,3 +1817,227 @@ Left _ -> return Nothing Right lang -> return $ Just lang) +parseCslJson :: CiteprocOutput a => Locale -> Text -> a +parseCslJson locale t = + case P.parseOnly + (P.many' (pCslJson locale) <* P.endOfInput) t of + Left _ -> fromText t + Right xs -> mconcat xs + +pCslJson :: CiteprocOutput a => Locale -> P.Parser a +pCslJson locale = P.choice + [ pCslText + , pCslQuoted + , pCslItalic + , pCslBold + , pCslUnderline + , pCslNoDecoration + , pCslSmallCaps + , pCslSup + , pCslSub + , pCslBaseline + , pCslNoCase + , pCslSymbol + ] + where + ((outerOpenQuote, outerCloseQuote), (innerOpenQuote, innerCloseQuote)) = + lookupQuotes locale + isSpecialChar c = c == '<' || c == '>' || c == '\'' || c == '"' || + c == '’' || (not (isAscii c) && (isQuoteChar c || isSuperscriptChar c)) + isQuoteChar = P.inClass + (T.unpack (outerOpenQuote <> outerCloseQuote <> + innerOpenQuote <> innerCloseQuote)) + isApostrophe '\'' = True + isApostrophe '’' = True + isApostrophe _ = False + pCsl = pCslJson locale + notFollowedBySpace = + P.peekChar' >>= guard . not . isSpaceChar + isSpaceChar = P.inClass [' ','\t','\n','\r'] + pOpenQuote = (("\"" <$ P.char '"') + <|> ("'" <$ P.char '\'') + <|> (outerCloseQuote <$ P.string outerOpenQuote) + <|> (innerCloseQuote <$ P.string innerOpenQuote)) + <* notFollowedBySpace + pSpace = P.skipWhile isSpaceChar + pCslText = fromText . addNarrowSpace <$> + ( do t <- P.takeWhile1 (\c -> isAlphaNum c && not (isSpecialChar c)) + -- apostrophe + P.option t $ do _ <- P.satisfy isApostrophe + t' <- P.takeWhile1 isAlphaNum + return (t <> "’" <> t') + <|> + P.takeWhile1 (\c -> not (isAlphaNum c || isSpecialChar c)) ) + pCslQuoted = addQuotes <$> + do cl <- pOpenQuote + mbc <- P.peekChar + case mbc of + Just c | T.singleton c == cl -> fail "unexpected close quote" + _ -> return () + mconcat <$> P.manyTill' pCsl (P.string cl) + pCslSymbol = do + c <- P.satisfy isSpecialChar + return $ + if isApostrophe c + then fromText "’" + else fromText $ T.singleton c + pCslItalic = addFontStyle ItalicFont . mconcat <$> + (P.string "<i>" *> P.manyTill' pCsl (P.string "</i>")) + pCslBold = addFontWeight BoldWeight . mconcat <$> + (P.string "<b>" *> P.manyTill' pCsl (P.string "</b>")) + pCslUnderline = addTextDecoration UnderlineDecoration . mconcat <$> + (P.string "<u>" *> P.manyTill' pCsl (P.string "</u>")) + pCslNoDecoration = addTextDecoration NoDecoration . mconcat <$> + (P.string "<span" *> pSpace *> + P.string "class=\"nodecor\"" *> pSpace *> P.char '>' *> + P.manyTill' pCsl (P.string "</span>")) + pCslSup = addVerticalAlign SupAlign . mconcat <$> + (P.string "<sup>" *> P.manyTill' pCsl (P.string "</sup>")) + pCslSub = addVerticalAlign SubAlign . mconcat <$> + (P.string "<sub>" *> P.manyTill' pCsl (P.string "</sub>")) + pCslBaseline = addVerticalAlign BaselineAlign . mconcat <$> + (P.string "<span" *> pSpace *> P.string "style=\"baseline\">" *> + P.manyTill' pCsl (P.string "</span>")) + pCslSmallCaps = addFontVariant SmallCapsVariant . mconcat <$> + ((P.string "<span" *> pSpace *> + P.string "style=\"font-variant:" *> pSpace *> + P.string "small-caps;" *> pSpace *> P.char '"' *> + pSpace *> P.char '>' *> P.manyTill' pCsl (P.string "</span>")) + <|> + (P.string "<sc>" *> P.manyTill' pCsl (P.string "</sc>"))) + pCslNoCase = addTextCase (localeLanguage locale) PreserveCase . mconcat <$> + (P.string "<span" *> pSpace *> + P.string "class=\"nocase\"" *> pSpace *> P.char '>' *> + P.manyTill' pCsl (P.string "</span>")) + addNarrowSpace = + T.replace " ;" "\x202F;" . + T.replace " ?" "\x202F?" . + T.replace " !" "\x202F!" . + T.replace " »" "\x202F»" . + T.replace "« " "«\x202F" + +lookupLocaleTerm :: Locale -> Text -> Maybe Text +lookupLocaleTerm locale termname = do + let terms = localeTerms locale + case M.lookup termname terms of + Just ((_,t):_) -> Just t + _ -> Nothing + +lookupQuotes :: Locale -> ((Text, Text), (Text, Text)) +lookupQuotes locale = ((outerOpen, outerClose), (innerOpen, innerClose)) + where + outerOpen = fromMaybe "\x201C" $ lookupLocaleTerm locale "open-quote" + outerClose = fromMaybe "\x201D" $ lookupLocaleTerm locale "close-quote" + innerOpen = fromMaybe "\x2018" $ lookupLocaleTerm locale "open-inner-quote" + innerClose = fromMaybe "\x2019" $ lookupLocaleTerm locale "close-inner-quote" + +isSuperscriptChar :: Char -> Bool +isSuperscriptChar c = M.member c superscriptChars + +superscriptChars :: M.Map Char Text +superscriptChars = M.fromList + [ ('\x00AA' , "\x0061") + , ('\x00B2' , "\x0032") + , ('\x00B3' , "\x0033") + , ('\x00B9' , "\x0031") + , ('\x00BA' , "\x006F") + , ('\x02B0' , "\x0068") + , ('\x02B1' , "\x0266") + , ('\x02B2' , "\x006A") + , ('\x02B3' , "\x0072") + , ('\x02B4' , "\x0279") + , ('\x02B5' , "\x027B") + , ('\x02B6' , "\x0281") + , ('\x02B7' , "\x0077") + , ('\x02B8' , "\x0079") + , ('\x02E0' , "\x0263") + , ('\x02E1' , "\x006C") + , ('\x02E2' , "\x0073") + , ('\x02E3' , "\x0078") + , ('\x02E4' , "\x0295") + , ('\x1D2C' , "\x0041") + , ('\x1D2D' , "\x00C6") + , ('\x1D2E' , "\x0042") + , ('\x1D30' , "\x0044") + , ('\x1D31' , "\x0045") + , ('\x1D32' , "\x018E") + , ('\x1D33' , "\x0047") + , ('\x1D34' , "\x0048") + , ('\x1D35' , "\x0049") + , ('\x1D36' , "\x004A") + , ('\x1D37' , "\x004B") + , ('\x1D38' , "\x004C") + , ('\x1D39' , "\x004D") + , ('\x1D3A' , "\x004E") + , ('\x1D3C' , "\x004F") + , ('\x1D3D' , "\x0222") + , ('\x1D3E' , "\x0050") + , ('\x1D3F' , "\x0052") + , ('\x1D40' , "\x0054") + , ('\x1D41' , "\x0055") + , ('\x1D42' , "\x0057") + , ('\x1D43' , "\x0061") + , ('\x1D44' , "\x0250") + , ('\x1D45' , "\x0251") + , ('\x1D46' , "\x1D02") + , ('\x1D47' , "\x0062") + , ('\x1D48' , "\x0064") + , ('\x1D49' , "\x0065") + , ('\x1D4A' , "\x0259") + , ('\x1D4B' , "\x025B") + , ('\x1D4C' , "\x025C") + , ('\x1D4D' , "\x0067") + , ('\x1D4F' , "\x006B") + , ('\x1D50' , "\x006D") + , ('\x1D51' , "\x014B") + , ('\x1D52' , "\x006F") + , ('\x1D53' , "\x0254") + , ('\x1D54' , "\x1D16") + , ('\x1D55' , "\x1D17") + , ('\x1D56' , "\x0070") + , ('\x1D57' , "\x0074") + , ('\x1D58' , "\x0075") + , ('\x1D59' , "\x1D1D") + , ('\x1D5A' , "\x026F") + , ('\x1D5B' , "\x0076") + , ('\x1D5C' , "\x1D25") + , ('\x1D5D' , "\x03B2") + , ('\x1D5E' , "\x03B3") + , ('\x1D5F' , "\x03B4") + , ('\x1D60' , "\x03C6") + , ('\x1D61' , "\x03C7") + , ('\x2070' , "\x0030") + , ('\x2071' , "\x0069") + , ('\x2074' , "\x0034") + , ('\x2075' , "\x0035") + , ('\x2076' , "\x0036") + , ('\x2077' , "\x0037") + , ('\x2078' , "\x0038") + , ('\x2079' , "\x0039") + , ('\x207A' , "\x002B") + , ('\x207B' , "\x2212") + , ('\x207C' , "\x003D") + , ('\x207D' , "\x0028") + , ('\x207E' , "\x0029") + , ('\x207F' , "\x006E") + , ('\x2120' , "\x0053\x004D") + , ('\x2122' , "\x0054\x004D") + , ('\x3192' , "\x4E00") + , ('\x3193' , "\x4E8C") + , ('\x3194' , "\x4E09") + , ('\x3195' , "\x56DB") + , ('\x3196' , "\x4E0A") + , ('\x3197' , "\x4E2D") + , ('\x3198' , "\x4E0B") + , ('\x3199' , "\x7532") + , ('\x319A' , "\x4E59") + , ('\x319B' , "\x4E19") + , ('\x319C' , "\x4E01") + , ('\x319D' , "\x5929") + , ('\x319E' , "\x5730") + , ('\x319F' , "\x4EBA") + , ('\x02C0' , "\x0294") + , ('\x02C1' , "\x0295") + , ('\x06E5' , "\x0648") + , ('\x06E6' , "\x064A") + ] diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.11/src/Citeproc.hs new/citeproc-0.12/src/Citeproc.hs --- old/citeproc-0.11/src/Citeproc.hs 2001-09-09 03:46:40.000000000 +0200 +++ new/citeproc-0.12/src/Citeproc.hs 2001-09-09 03:46:40.000000000 +0200 @@ -37,13 +37,13 @@ rCitations = map ( trimR . localizeQuotes locale . movePunct - . renderOutput opts + . renderOutput opts locale ) citationOs rBibliography = map (second ( trimR . localizeQuotes locale . movePunct - . renderOutput opts{ linkCitations = False } )) + . renderOutput opts{ linkCitations = False } locale )) bibliographyOs locale = mergeLocales mblang style trimR = dropTextWhileEnd (== ' ') diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.11/test/Spec.hs new/citeproc-0.12/test/Spec.hs --- old/citeproc-0.11/test/Spec.hs 2001-09-09 03:46:40.000000000 +0200 +++ new/citeproc-0.12/test/Spec.hs 2001-09-09 03:46:40.000000000 +0200 @@ -186,12 +186,14 @@ then do modify $ \st -> st{ passed = (category test, path test) : passed st } -- suppress PASSED messages - -- liftIO $ TIO.putStrLn $ "[PASSED] " <> path test + -- liftIO $ TIO.putStrLn $ "[PASSED] " <> path test return Passed else do modify $ \st -> st{ failed = (category test, path test) : failed st } liftIO $ do - TIO.putStrLn $ "[FAILED] " <> T.pack (path test) + TIO.putStrLn $ (if path test `elem` expectedFailures + then "[FAILED:EXPECTED] " + else "[FAILED:UNEXPECTED] ") <> T.pack (path test) showDiff expected actual return $ Failed actual expected @@ -312,7 +314,7 @@ putStrLn "" let categories = sort $ Set.toList $ foldr (Set.insert . category) mempty testCases - putStrLn $ printf "%-15s %6s %6s %6s %6s" + putStrLn $ printf "%-30s %6s %6s %6s %6s" ("CATEGORY" :: String) ("PASS" :: String) ("FAIL" :: String) @@ -324,17 +326,17 @@ let e = length . filter ((== cat) . fst) . errored $ counts let s = length . filter ((== cat) . fst) . skipped $ counts let percent = (fromIntegral p / fromIntegral (p + f + e) :: Double) - putStrLn $ printf "%-15s %6d %6d %6d %6d |%-20s|" + putStrLn $ printf "%-30s %6d %6d %6d %6d |%-20s|" (T.unpack cat) p f e s (replicate (floor (percent * 20.0)) '+') mapM_ resultsFor categories - putStrLn $ printf "%-15s %6s %6s %6s %6s" + putStrLn $ printf "%-30s %6s %6s %6s %6s" ("-------------" :: String) ("-----" :: String) ("-----" :: String) ("-----" :: String) ("-----" :: String) - putStrLn $ printf "%-15s %6d %6d %6d %6d" + putStrLn $ printf "%-30s %6d %6d %6d %6d" ("(all)" :: String) (length (passed counts)) (length (failed counts)) @@ -453,5 +455,7 @@ "test/csl/bugreports_AutomaticallyDeleteItemsFails.txt", "test/csl/affix_WithCommas.txt", "test/csl/affix_CommaAfterQuote.txt", - "test/overrides/flipflop_NumericField.txt" + "test/overrides/flipflop_NumericField.txt", + "test/csl/testers_SecondAutoGeneratedZoteroPluginTest.txt", + "test/csl/testers_FirstAutoGeneratedZoteroPluginTest.txt" ] diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.11/test/extra/issue_173.txt new/citeproc-0.12/test/extra/issue_173.txt --- old/citeproc-0.11/test/extra/issue_173.txt 1970-01-01 01:00:00.000000000 +0100 +++ new/citeproc-0.12/test/extra/issue_173.txt 2001-09-09 03:46:40.000000000 +0200 @@ -0,0 +1,66 @@ +>>===== MODE =====>> +bibliography +<<===== MODE =====<< + + + + +>>===== RESULT =====>> +<div class="csl-bib-body"> + <div class="csl-entry">https://www.researchgate.net/publication/308399994_Teo's_Brew_-_The_Re-composition_of_'Bitches_Brew'_in_the_Studio</div> +</div> +<<===== RESULT =====<< + + +>>===== CSL =====>> +<?xml version="1.0" encoding="utf-8"?> +<style + xmlns="http://purl.org/net/xbiblio/csl"; + class="note" + version="1.0"> + <info> + <id /> + <title /> + </info> + <citation> + <layout> + <text variable="URL"/> + </layout> + </citation> + <bibliography> + <layout> + <text variable="URL"/> + </layout> + </bibliography> +</style> +<<===== CSL =====<< + + +>>===== CITATION-ITEMS =====>> +[ [ {"id":"Doe" } ] ] +<<===== CITATION-ITEMS =====<< + + + +>>===== INPUT =====>> +[ + { + "author": [ + { + "family": "Doe", + "given": "John" + } + ], + "id": "Doe", + "type": "book", + "title": "Title", + "URL":"https://www.researchgate.net/publication/308399994_Teo's_Brew_-_The_Re-composition_of_'Bitches_Brew'_in_the_Studio" + } +] +<<===== INPUT =====<< + + +>>===== VERSION =====>> +1.0 +<<===== VERSION =====<< + diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.11/test/extra/issue_174.txt new/citeproc-0.12/test/extra/issue_174.txt --- old/citeproc-0.11/test/extra/issue_174.txt 1970-01-01 01:00:00.000000000 +0100 +++ new/citeproc-0.12/test/extra/issue_174.txt 2001-09-09 03:46:40.000000000 +0200 @@ -0,0 +1,98 @@ +>>===== MODE =====>> +citation +<<===== MODE =====<< + + +>>===== RESULT =====>> +Title, ALCT, 3 +<<===== RESULT =====<< + + +>>===== CSL =====>> +<?xml version="1.0" encoding="utf-8"?> +<style class="note" version="1.0" xmlns="http://purl.org/net/xbiblio/csl";> + <info> + <title>Variable substitution test</title> + <id>http://www.zotero.org/styles/subsitute-test</id> + </info> + <citation> + <layout> + <group delimiter=", "> + <!-- short title --> + <names variable="composer"> + <substitute> + <text form="short" variable="title"/> + </substitute> + </names> + <text form="short" variable="title"/> + <!-- short container title --> + <names variable="author"> + <substitute> + <text form="short" variable="container-title"/> + </substitute> + </names> + <text form="short" variable="container-title"/> + <!-- number --> + <names variable="illustrator"> + <substitute> + <number variable="volume"/> + </substitute> + </names> + <number variable="volume"/> + </group> + </layout> + </citation> + <bibliography> + <layout> + <group delimiter=", "> + <!-- title --> + <names variable="composer"> + <substitute> + <text variable="title"/> + </substitute> + </names> + <text variable="title"/> + <!-- container title --> + <names variable="author"> + <substitute> + <text variable="container-title"/> + </substitute> + </names> + <text variable="container-title"/> + <!-- number as text --> + <names variable="illustrator"> + <substitute> + <text variable="volume"/> + </substitute> + </names> + <text variable="volume"/> + </group> + </layout> + </bibliography> +</style> +<<===== CSL =====<< + + +>>===== CITATION-ITEMS =====>> +[ [ {"id":"test" } ] ] +<<===== CITATION-ITEMS =====<< + + + +>>===== INPUT =====>> +[ + { "id": "test", + "title": "A title with a long form", + "title-short": "Title", + "container-title": "A long container title", + "container-title-short": "ALCT", + "volume": "3" + } +] +<<===== INPUT =====<< + + +>>===== VERSION =====>> +1.0 +<<===== VERSION =====<< + diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/citeproc-0.11/test/extra/issue_175.txt new/citeproc-0.12/test/extra/issue_175.txt --- old/citeproc-0.11/test/extra/issue_175.txt 1970-01-01 01:00:00.000000000 +0100 +++ new/citeproc-0.12/test/extra/issue_175.txt 2001-09-09 03:46:40.000000000 +0200 @@ -0,0 +1,79 @@ +>>===== MODE =====>> +citation +<<===== MODE =====<< + + +>>===== RESULT =====>> +numeric volume: volumes 1–2 +<<===== RESULT =====<< + + +>>===== CSL =====>> +<?xml version="1.0" encoding="utf-8"?> +<style class="note" version="1.0" xmlns="http://purl.org/net/xbiblio/csl";> + <info> + <title>Numeric test</title> + <id>http://www.zotero.org/styles/numeric-test</id> + <link href="http://www.zotero.org/styles/locator-test"; rel="self"/> + <updated>2025-05-20T00:00:00+00:00</updated> + </info> + <citation> + <layout> + <group delimiter=" "> + <choose> + <if variable="volume"> + <choose> + <if is-numeric="volume"> + <text value="numeric volume:"/> + </if> + <else> + <text value="non-numeric volume:"/> + </else> + </choose> + <label variable="volume"/> + <text variable="volume"/> + </if> + </choose> + <choose> + <if variable="locator"> + <choose> + <if is-numeric="locator"> + <text value="numeric locator:"/> + </if> + <else> + <text value="non-numeric locator:"/> + </else> + </choose> + <label variable="locator"/> + <text variable="locator"/> + </if> + </choose> + </group> + </layout> + </citation> +</style> +<<===== CSL =====<< + + +>>===== CITATION-ITEMS =====>> +[ [ {"id":"VolumeTest1" } ] ] +<<===== CITATION-ITEMS =====<< + + + +>>===== INPUT =====>> +[ + { + "id": "VolumeTest1", + "type": "book", + "title": "Title", + "volume": "1–2" + } +] +<<===== INPUT =====<< + + +>>===== VERSION =====>> +1.0 +<<===== VERSION =====<< +
