Pre-Formed Regular Expressions

Prior Reading <ref target="doc:learn_regex">Introduction to Regular Expressions</ref>

Regex for All Edition Components

The regular expressions in this section are useful in multiple edition components (e.g., you may use them in annotations and critical paratexts).

Rationale Find Replace With Replaces quotation marks with q in converted files. "([\w][^"]+)"(([\W])|($)) (Note: constrain to text() in the XPath context) <q>$1</q>$2 LEMDO uses en dashes in date and number ranges. We frequently receive files from editors that contain hyphens. This converts such hyphens to en dashes (without converting the hyphens that occur in the dates that are the values of attributes). (\d+)-(\d+) (Note: constrain to text() in the XPath context) $1–$2

Regex for Semi-Diplomatic Transcriptions

The regular expressions in this section are used in semi-diplomatic transcriptions that were converted from IML.

Rationale Find Replace With When you are working with IML-TEI converted texts, you may come across a lot of situations where an lb element appears right inside the ab of a speech, instead of before the sp tag. This removes those unwanted lb elements. Note: we recommend running this regex before adding renditions or making other changes to the file. (<sp>\s+<speaker>.+?</speaker>\s*<ab>\s*)(<lb/>) (Note: make sure you check Dot matches all) $2$1 This removes lb elements with n giving editorial line numbers. <lb\s+n="\d*\.*\d*"/> Leave empty When you convert a file from TCP or IML to TEI, there will likely be old facsimile links in the pb element. This removes old facsimile links. (<pb)\sfacs=".+?"(/>) $1$2 You may come across instances of the lb element appearing after sp rather than before it. This corrects order of elements. (<sp>\s+)(<lb/>\s*)(<speaker>.+?</speaker>) $2$1$3 Standardizes attribute order on forme works. Note: this regex should be run before running any other regex on forme works. <fw(\srendition="(\s*\w+:\w+)+")(\stype="\w+") Explanation of find regex: this regex has three backreference groups: (\srendition="(\s*\w+:\w+)+"): contains the rendition attribute and its value(s) (\s*\w+:\w+)+: accounts for one or more values on the rendition attribute (note that format for rendition values is rnd:[value]) (\stype="\w+"): contains the type attribute and its value <fw$3$1 Explanation of replace regex: by putting $3 before $1 in the replace, this regex places the type attribute before the rendition attribute, which will format the fw elements correctly for other regex. LEMDO’s default styling is to centre signature numbers. This removes unnecessary rendition attributes that only have the value rnd:centre from signature numbers. (<fw type="sig")\srendition="rnd:centre" $1 LEMDO’s default styling is to align catch words to the right. This removes unnecessary rendition attributes that only have the value rnd:right from catch words. (<fw type="catch")\srendition="rnd:right" $1 Removes unnecessary space from between the letter and number of signature marks (<fw type="sig">[a-zA-Z]+)\s(\d</fw>) Explanation of find regex: this find has two backreference groups that are kept in the conversion: (<fw type="sig">[a-zA-Z]+): contains the opening fw tag, all of the attributes on it, and the letter of the signature mark. (\d</fw>): contains the number of the signature mark and the closing /fw tag. <fw type="sig"> … </fw>: constrains the search to text encoded as signature marks. [a-zA-Z]+: indicates the letter or letters in the signature mark. They may be capital or lower case letters, and there will be at least one (there may be more than one in long books with lots of gatherings). \s: indicates the space between the letter and number. This will be removed in the conversion. \d: indicates the number in the signature mark. $1$2 Explanation of replace regex: $1: Keeps the first backreference group during the conversion process. $2: Keeps the second backreference group during the conversion process. By excluding the \s from the replace, we eliminate the unwanted space. Removes italic and centre tagging from running titles. Note: after running this, always also run the regex to remove empty rendition attributes. (<fw\stype="runningTitle"\srendition=")((\s*rnd:centre)|(\s*rnd:italic)|(\s*(rnd:\w+)))*" $1$6" Removes empty rendition attributes. Note: run this after using regex to remove values from rendition attributes. \srendition="" Leave empty Removes italic and centre tagging on hi elements in running titles. Note: after running this, always also run the regex to remove empty hi elements. (<hi\srendition=")((\s*rnd:centre)|(\s*rnd:italic)|(\s*(rnd:\w+)))*" (Note: constrain to //fw[contains(@type, 'runningTitle')] in the XPath field) $1$6 Removes empty hi elements after getting rid of values. Note: run this after using regex to remove values from rendition attributes on the hi element. <hi\srendition="">(.+?)<\hi> $1 Removes italic tagging on speaker elements. Note: after running this, always also run the regex to remove empty rendition attributes. (<speaker\srendition=")((\s*rnd:italic)|(\s*(\w+:\w+)))*" $1$5" Removes italic tagging on hi elements in speech prefixes and adds hi elements with rnd:normal around characters in roman type. Note: after running this, always also run the regex to remove empty hi elements and the regex to remove hi elements with empty text nodes. (<speaker>)(.+?)*(<hi\srendition=")((\s*rnd:italic)|(\s*(\w+:\w+)))*">(.+?)*(</speaker>) $1<hi rendition="rnd:normal">$2</hi>$3<hi rendition="rnd:normal">$4</hi>$5 Removes hi element with empty text nodes. <hi\srendition=".+?"></hi> Leave empty Standardizes the order of attributes on the stage element. Note: run this regex before running any other regex on stage directions. (<stage)(\srendition="(\s*(\w+:\w+))+")(\stype="(\s*\w+)+") $1$5$2 Removes italic tagging on stage elements. Note: after running this, always also run the regex to remove empty rendition attributes. (<hi\srendition=")((\s*rnd:italic)|(\s*(rnd:\w+)))*" (Note: constrain to //stage in the XPath field) $1$5 Removes italic tagging using the hi element in stage directions. Note: after running this, always also run the regex to remove empty hi elements. (<hi\srendition=")((\s*rnd:italic)|(\s*(rnd:\w+)))*" (Note: constrain to //stage in the XPath field) $1$5" Removes who attributes <sp who="#\w+"> Explanation of find regex: <sp>: Constrains the search to sp elements. who="#\w+": Searches for the who attribute. This is the section that will be removed during the conversion. The hash character indicates that every value for the who attribute is prefixed by a hash character. \w indicates that the value following the hash character may consist of alphanumeric characters and underscores (word characters). The plus sign indicates that there will be one or more word character. <sp> Removes rnd:justify. Note: after running this, always also run the regex to remove empty hi elements. \srendition="rnd:justify" Leave empty Removes supplied page numbers on the n attribute on the pb element (<pb\sn=")\d+;\s(\w+"/>) $1$2 The XSLT conversion to programmatically number lines in semi-diplomatic transcriptions puts the n attribute before the type attribute. While this is valid XML, LEMDO opts for the type attribute to appear before the n attribute. Consistency ensures easy searchability across our repository. This regex reorders the attributes so that they are consistent. (<lb)(\sn="\d+")(\stype="wln")(/>) $1$3$2$4

Regex for Modernized Texts

The regular expressions in this section are used in modernized texts that were converted from IML.

Rationale Find Replace With Adds n attribute to div elements <div type="(\w+)" xml:id="([a-zA-Z]+_\w*)_([a-z])(\d+)"> <div type="$1" n="$4" xml:id="$2_$3$4">

Other Resources LEMDO YouTube video: Modernization (Technical): Regular Expressions Jan Goyvaerts’s Regular-Expressions.info