Preserving uppercase with \MakeSentenceCase for signs in non-ascii range with biblatex, biber, and pdflatex

[Erebus Mons]

Dear all,

I experience unexpected behavior with biblatex, biber, and pdflatex (xelatex is not affected), concerning \MakeSentenceCase.

In bibentries where I want to preserve non-initial uppercase with a pair of curly braces, this only works for characters in ascii-range, but not for random unicode examples (illustrated below for é and ü). However, with double curly braces, uppercase is preserved.

I am using TexLive 2019 (on Linux, if this is important).

On overleaf, doubling the curly braces has no effect.

I assume that this is not expected behavior. If this is a bug, to whom should this be reported?

Here is the MWE:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\listfiles
\documentclass{article}

\usepackage{biblatex}
\DeclareFieldFormat[article]{titlecase}{\MakeSentenceCase*{#1}}%
\usepackage{filecontents}

\begin{filecontents}{jobname.bib}
@ARTICLE{a1,
AUTHOR = {Doe, John},
DATE = {1996},
JOURNALTITLE = {Journal of Stuff},
NUMBER = {2},
PAGES = {1--10},
TITLE = {This? {T}hat? --- {S}trange World},
VOLUME = {15},
}
@ARTICLE{a2,
AUTHOR = {Doe, John},
DATE = {1996},
JOURNALTITLE = {Journal of Stuff},
NUMBER = {3},
PAGES = {1--10},
TITLE = {Ceci? {C}ela? --- {É}trange},
VOLUME = {15},
}
@ARTICLE{a3,
AUTHOR = {Doe, John},
DATE = {1996},
JOURNALTITLE = {Journal of Curly Stuff},
NUMBER = {2},
PAGES = {1--10},
TITLE = {Dies? {D}as? --- {{Ü}}berflüssig},
VOLUME = {15},
}
@ARTICLE{a4,
AUTHOR = {Doe, John},
DATE = {1996},
JOURNALTITLE = {Journal of Stuff},
NUMBER = {2},
PAGES = {1--10},
TITLE = {Dies? {D}as? --- {Ü}berflüssig},
VOLUME = {15},
}
\end{filecontents}

\addbibresource{jobname.bib}

\begin{document}

Have a good look at the beginning of the word after the hyphen.

\nocite{*}

\printbibliography

\end{document}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Here is the output of \listfiles:

*File List*
article.cls 2018/09/03 v1.4i Standard LaTeX document class
size10.clo 2018/09/03 v1.4i Standard LaTeX file (size option)
biblatex.sty 2018/11/02 v3.12 programmable bibliographies (PK/MW)
pdftexcmds.sty 2018/09/10 v0.29 Utility functions of pdfTeX for LuaTeX (HO)
infwarerr.sty 2016/05/16 v1.4 Providing info/warning/error messages (HO)
ifluatex.sty 2016/05/16 v1.4 Provides the ifluatex switch (HO)
ltxcmds.sty 2016/05/16 v1.23 LaTeX kernel commands for general use (HO)
ifpdf.sty 2018/09/07 v3.3 Provides the ifpdf switch
etoolbox.sty 2018/08/19 v2.5f e-TeX tools for LaTeX (JAW)
keyval.sty 2014/10/28 v1.15 key=value parser (DPC)
kvoptions.sty 2016/05/16 v3.12 Key value format for package options (HO)
kvsetkeys.sty 2016/05/16 v1.17 Key value parser (HO)
etexcmds.sty 2016/05/16 v1.6 Avoid name clashes with e-TeX commands (HO)
logreq.sty 2010/08/04 v1.0 xml request logger
logreq.def 2010/08/04 v1.0 logreq spec v1.0
ifthen.sty 2014/09/29 v1.1c Standard LaTeX ifthen package (DPC)
url.sty 2013/09/16 ver 3.4 Verb mode for urls, etc.
blx-dm.def 2018/11/02 v3.12 biblatex localization (PK/MW)
blx-compat.def 2018/11/02 v3.12 biblatex compatibility (PK/MW)
biblatex.def 2018/11/02 v3.12 biblatex compatibility (PK/MW)
standard.bbx 2018/11/02 v3.12 biblatex bibliography style (PK/MW)
numeric.bbx 2018/11/02 v3.12 biblatex bibliography style (PK/MW)
numeric.cbx 2018/11/02 v3.12 biblatex citation style (PK/MW)
biblatex.cfg
filecontents.sty 2018/05/30 v1.4 Create an external file from within a LaTeX
document
english.lbx 2018/11/02 v3.12 biblatex localization (PK/MW)
mwe2.bbl
***********

Best,

Erebus

[jon]

Post by Erebus Mons
Dear all,
I experience unexpected behavior with biblatex, biber, and pdflatex (xelatex is not affected), concerning \MakeSentenceCase.
In bibentries where I want to preserve non-initial uppercase with a pair of curly braces, this only works for characters in ascii-range, but not for random unicode examples (illustrated below for é and ü). However, with double curly braces, uppercase is preserved.
I am using TexLive 2019 (on Linux, if this is important).
On overleaf, doubling the curly braces has no effect.
I assume that this is not expected behavior. If this is a bug, to whom should this be reported?
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\listfiles
\documentclass{article}
\usepackage{biblatex}
\DeclareFieldFormat[article]{titlecase}{\MakeSentenceCase*{#1}}%
\usepackage{filecontents}
\begin{filecontents}{jobname.bib}
@ARTICLE{a1,
AUTHOR = {Doe, John},
DATE = {1996},
JOURNALTITLE = {Journal of Stuff},
NUMBER = {2},
PAGES = {1--10},
TITLE = {This? {T}hat? --- {S}trange World},
VOLUME = {15},
}
@ARTICLE{a2,
AUTHOR = {Doe, John},
DATE = {1996},
JOURNALTITLE = {Journal of Stuff},
NUMBER = {3},
PAGES = {1--10},
TITLE = {Ceci? {C}ela? --- {É}trange},
VOLUME = {15},
}
@ARTICLE{a3,
AUTHOR = {Doe, John},
DATE = {1996},
JOURNALTITLE = {Journal of Curly Stuff},
NUMBER = {2},
PAGES = {1--10},
TITLE = {Dies? {D}as? --- {{Ü}}berflüssig},
VOLUME = {15},
}
@ARTICLE{a4,
AUTHOR = {Doe, John},
DATE = {1996},
JOURNALTITLE = {Journal of Stuff},
NUMBER = {2},
PAGES = {1--10},
TITLE = {Dies? {D}as? --- {Ü}berflüssig},
VOLUME = {15},
}
\end{filecontents}
\addbibresource{jobname.bib}
\begin{document}
Have a good look at the beginning of the word after the hyphen.
\nocite{*}
\printbibliography
\end{document}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
*File List*
article.cls 2018/09/03 v1.4i Standard LaTeX document class
size10.clo 2018/09/03 v1.4i Standard LaTeX file (size option)
biblatex.sty 2018/11/02 v3.12 programmable bibliographies (PK/MW)
pdftexcmds.sty 2018/09/10 v0.29 Utility functions of pdfTeX for LuaTeX (HO)
infwarerr.sty 2016/05/16 v1.4 Providing info/warning/error messages (HO)
ifluatex.sty 2016/05/16 v1.4 Provides the ifluatex switch (HO)
ltxcmds.sty 2016/05/16 v1.23 LaTeX kernel commands for general use (HO)
ifpdf.sty 2018/09/07 v3.3 Provides the ifpdf switch
etoolbox.sty 2018/08/19 v2.5f e-TeX tools for LaTeX (JAW)
keyval.sty 2014/10/28 v1.15 key=value parser (DPC)
kvoptions.sty 2016/05/16 v3.12 Key value format for package options (HO)
kvsetkeys.sty 2016/05/16 v1.17 Key value parser (HO)
etexcmds.sty 2016/05/16 v1.6 Avoid name clashes with e-TeX commands (HO)
logreq.sty 2010/08/04 v1.0 xml request logger
logreq.def 2010/08/04 v1.0 logreq spec v1.0
ifthen.sty 2014/09/29 v1.1c Standard LaTeX ifthen package (DPC)
url.sty 2013/09/16 ver 3.4 Verb mode for urls, etc.
blx-dm.def 2018/11/02 v3.12 biblatex localization (PK/MW)
blx-compat.def 2018/11/02 v3.12 biblatex compatibility (PK/MW)
biblatex.def 2018/11/02 v3.12 biblatex compatibility (PK/MW)
standard.bbx 2018/11/02 v3.12 biblatex bibliography style (PK/MW)
numeric.bbx 2018/11/02 v3.12 biblatex bibliography style (PK/MW)
numeric.cbx 2018/11/02 v3.12 biblatex citation style (PK/MW)
biblatex.cfg
filecontents.sty 2018/05/30 v1.4 Create an external file from within a LaTeX
document
english.lbx 2018/11/02 v3.12 biblatex localization (PK/MW)
mwe2.bbl
***********
Best,
Erebus

i believe you should still report bugs here:

http://github.com/plk/biblatex

although this does not address the specific concern you raise, the
point of \MakeSentenceCase* (the starred version) is to be sensitive
to language of the entry. so i think the biblatex 'way' is to add a
field like:

langid = {german},

to the entry rather than wrap the word in braces.

as an aside, it is usually better to wrap the whole word in braces
rather than just the first letter, which would interfere with the
kerning between the first letter and the rest of the word. (insert an
superfluous pun here about strange that looks.)

cheers,
jon.

[Erebus Mons]

Post by jon
although this does not address the specific concern you raise, the
point of \MakeSentenceCase* (the starred version) is to be sensitive
to language of the entry. so i think the biblatex 'way' is to add a
langid = {german},
to the entry rather than wrap the word in braces.

Adding the langid does remove the issue, and since \MakeSentenceCase* should not be tinkered with, it is probably the simplest way of avoiding the problem.

For reference, here is the working minimal example:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\documentclass{article}

\usepackage{biblatex}
\DeclareFieldFormat[article]{titlecase}{\MakeSentenceCase*{#1}}%
\usepackage{filecontents}

\begin{filecontents}{jobname.bib}
@ARTICLE{a1,
AUTHOR = {Doe, John},
DATE = {1996},
JOURNALTITLE = {Journal of Stuff},
NUMBER = {2},
PAGES = {1--10},
TITLE = {This? {T}hat? --- {Strange} World},
VOLUME = {15},
}
@ARTICLE{a2,
AUTHOR = {Doe, John},
DATE = {1996},
JOURNALTITLE = {Journal of Stuff},
NUMBER = {3},
PAGES = {1--10},
LANGID = {french},
TITLE = {Ceci? {C}ela? --- {Étrange}},
VOLUME = {15},
}
@ARTICLE{a3,
AUTHOR = {Doe, John},
DATE = {1996},
JOURNALTITLE = {Journal of Curly Stuff},
NUMBER = {2},
PAGES = {1--10},
LANGID = {german},
TITLE = {Dies? {D}as? --- {{Überflüssig}}},
VOLUME = {15},
}
@ARTICLE{a4,
AUTHOR = {Doe, John},
DATE = {1996},
JOURNALTITLE = {Journal of Stuff},
NUMBER = {2},
PAGES = {1--10},
LANGID = {german},
TITLE = {Dies? {D}as? --- {Überflüssig}},
VOLUME = {15},
}
\end{filecontents}

\addbibresource{jobname.bib}

\begin{document}

Uppercase protected, just as it should be.

\nocite{*}

\printbibliography

\end{document}

Post by jon
as an aside, it is usually better to wrap the whole word in braces
rather than just the first letter, which would interfere with the
kerning between the first letter and the rest of the word. (insert an
superfluous pun here about strange that looks.)

Point well taken.

Best,

Erebus

Post by jon
cheers,
jon.