• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    METHOD AND SYSTEM FOR CONVERTING A COMPUTER-READABLE DOCUMENT AND MEDIUM. The present invention relates to a formula detection mechanism and associated method. The formula detection engine finds formulas within a fixed format document part by identifying formula seeds. The formula detection engine creates and expands an outline around the formula seed to define a formula area. To eliminate overlap with surrounding normal text, the formula area is divided into multiple formula areas based on vertical position and horizontal spacing between formula elements. After being ordered vertically, areas of formulas overlapping horizontally are merged to rebuild the formula as a capable element for flow.
    公开号:BR112014017832B1
    申请号:R112014017832-1
    申请日:2012-01-23
    公开日:2021-07-06
    发明作者:Milos Lazarevic；Milos Raskovic；Aljosa Obuljen；Vanja Petrovic Tankovic
    申请人:Microsoft Technology Licensing, Llc；
    IPC主号:
    专利说明:

    BACKGROUND
    [001] Stream-format documents and fixed-format documents are widely used and have different purposes. Flow format documents organize a document using complex logical formatting structures such as sections, paragraphs, columns and tables. As a result, stream-format documents offer flexibility and easy modification making them suitable for tasks involving documents that are frequently updated or subject to significant editing. In contrast, fixed-format documents organize a document using basic physical layout elements such as plain text, paths and images to preserve the appearance of the original. Fixed format documents offer consistent and accurate format layout making them suitable for tasks involving documents that are not changed frequently or extensively or where uniformity is desired. Examples of such tasks include archival document, high-quality reproduction, and font files for commercial printing and publishing. Fixed-format documents are often created from stream-format source documents. Fixed-format documents also include digital reproductions (eg, scanning and photos) of physical documents (ie, paper).
    [002] In situations where editing a document in fixed format is desired, but the source document in stream format is not available, the document in fixed format must be converted to a document in stream format. Conversion involves parsing the fixed-format document and transforming the basic physical layout elements of the fixed-format document into the more complex logical elements used in a stream-format document. Existing document converters encountering complex elements, such as formulas and mathematical expressions, resort to basic techniques designed to preserve visual fidelity of fixed-format document layout (eg, text frames, line spacing, character spacing, and images ) with the cost of the flow of the output document. The result is a limited stream format document that requires the user to perform substantial manual reconstruction to have a truly useful stream format document. It is in relation to these and other considerations that the present invention has been embodied. SUMMARY
    [003] The following Summary is provided to introduce a selection of concepts in a simplified form, which are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
    [004] An embodiment of the formula detection engine performs the associated formula detection method as part of the conversion process transforming a fixed format document to a stream format document. The formula detection engine creates the initial formula areas by identifying elements that are potentially part of a mathematical formula and grouping those elements into formula areas based on the relative position of those elements. The formula detection engine starts by identifying formula seeds in the parsed elements. A formula seed is a text element that carries some indication of being part of a formula, such as plain text recorded with fonts used exclusively, or almost exclusively, to display mathematical expressions and mathematical operators, symbols or keywords, which are used exclusively, or almost exclusively, in mathematical formulas. Once formula seeds are identified, the formula detection engine defines a formula area (ie a contour) around each of the detected formula seeds and expands the contours to group formula seeds and others elements based on proximity. All page elements enclosed by the formula area confinement box are assumed to be the captured elements.
    [005] Next, the formula detection engine eliminates overlap between formula areas and any surrounding normal text (that is, text elements that are not part of a mathematical formula) by subdividing formula areas by overlapping normal text based on vertical position and divide formula areas based on horizontal spacing. The formula detection engine starts by analyzing each formula area to determine whether or not the formula area overlaps any areas of normal text. If the formula area overlaps any normal text, the formula detection engine splits the formula area by grouping captured elements based on vertical position. A new formula area is created around each group of captured elements. The formula detection engine analyzes the contents of new formula areas to overlap with normal text, and any new formula areas still containing normal text are further split. Once the formula areas are split, the formula detection engine splits each formula area according to the horizontal spacing between the captured elements.
    [006] Finally, the formula detection engine reconstructs mathematical formulas as capable elements for flow by merging any neighboring formula areas based on proximity. The formula detection engine uses information about the positions of neighboring text elements to prevent merging of formula areas appearing on different lines on a page. Areas of formulas that overlap horizontally (that is, are at least partially vertically aligned) and are within a selected vertical separation distance are grouped as merge candidates. The formula detection engine sorts (for example, sorts) the merge candidates according to the vertical separation between the formula areas and replaces the two formula areas of the merge candidate with a new formula area, as long as both formula areas have not already been merged and are not on separate lines.
    [007] Details of one or more modalities are shown in the attached drawings and in the description below. Other features and advantages will be apparent from a reading of the detailed description below and an analysis of the associated drawings. It is to be understood that the following detailed description is explanatory only and is not restrictive of the invention as claimed. BRIEF DESCRIPTION OF THE DRAWINGS
    [008] Additional features, aspects and advantages will become better understood by reference to the following detailed description, appended claims and appended Figures, where elements are not to scale in order to show details more clearly, where equal reference numbers indicate like elements throughout the various views, and wherein: Figure 1 illustrates a system including the formula detection mechanism; Figure 2 is a block diagram showing the operational flow of an embodiment of the document processor; Figure 3 is a flowchart showing an embodiment of the formula detection method; Figure 4 is a flowchart showing an embodiment of the process for creating the initial formula areas used in the formula detection method; Figure 5A graphically illustrates an exemplary operation of identifying formula seeds applied to a selected portion of parsed data from a fixed-format document that contains several formulas and mathematical equations appearing within a normal text development; Figure 5B graphically illustrates an exemplary operation of an embodiment of the process of creating the initial formula areas; Figure 6 is a flowchart showing an embodiment of the process for eliminating overlap between formula areas and plain text used in the formula detection method; Figures 7A-7D graphically illustrate selected exemplary operations of an embodiment of the process of eliminating overlap between formula areas and normal text used in an embodiment of the formula detection method; Figure 8 is a flowchart showing an embodiment of the process of reconstructing the individual formulas used in the formula detection method; Figures 9A-9B graphically illustrate selected exemplary operations of one embodiment of the process of reconstructing the individual formulas used in the formula detection method; Figure 10 illustrates an exemplary computing device performing an embodiment of the formula detection mechanism; Figure 11 is a simplified block diagram of an exemplary computing device suitable for practicing embodiments of the formula detection mechanism; Figure 12A illustrates an embodiment of a mobile computing device performing an embodiment of the formula detection mechanism; Figure 12B is a simplified block diagram of an exemplary mobile computing device suitable for practicing embodiments of the formula detection mechanism; and Figure 13 is a simplified block diagram of an exemplary distributed computing system suitable for practicing embodiments of the formula detection mechanism. DETAILED DESCRIPTION
    [009] A formula detection mechanism and associated method for identifying formulas and mathematical expressions in data extracted from a document in fixed format are described in this document and illustrated in the attached Figures. The formula detection engine finds formulas within the document part in fixed format by identifying a formula seed. The formula detection engine creates and expands an outline around the formula seed to define a formula area. To eliminate overlap with surrounding normal text, the formula area is divided into multiple formula areas based on vertical position and horizontal spacing between captured elements. The resulting formula areas ordered vertically and formula areas overlapping horizontally are merged to rebuild the formula as a capable element for flow.
    [010] Figure 1 illustrates an embodiment of a system incorporating formula detection mechanism 100. In the illustrated embodiment, formula detection mechanism 100 operates as part of a document converter 102 running on a computing device 104. document converter 102 converts a fixed format document 106 to a stream format document 108 using a parser 110, a document processor 112 and a serializer 114. The parser 110 reads and extracts data from the fixed format document 106. Data extracted from the document in fixed format is recorded in a data store 116 accessible by document processor 112 and serializer 114. Document processor 112 analyzes and transforms the data into elements capable of streaming using one or more detection and /or reconstruction (e.g., the formula 100 detection mechanism of the present invention). Finally, the serializer 114 writes the stream-capable elements into a stream-capable document format (eg, a word processing format).
    [011] Figure 2 illustrates a modality of the operational flow of the document processor 112 with more details. Document processor 112 includes an optional optical character recognition (OCR) engine 202, a layout analysis engine 204, and a semantic analysis engine 206. The data contained in data store 116 includes physical layout objects 208 and logical layout objects 210. In some embodiments, physical layout objects 208 and logical layout objects 210 are arranged hierarchically in an arrangement such as a group tree (ie, data objects). In various embodiments, a page is the top-level group for the 208 physical layout objects, while a section is the top-level group for the 210 logical layout objects. general are stored as physical layout objects 208 arranged by the containing page in the document in fixed format 106. Basic physical layout objects include plain text, images, and paths. Running text is the text elements in page content streams specifying the positions where characters are drawn when displaying the document in fixed format. Images are the trace images (ie, scenes) stored in the document in fixed format 106. Paths describe elements such as lines, curves (eg, cubic Bezier curves), and text outlines used to construct vector graphics. Logical data objects include flow-able elements such as sections, paragraphs, columns, and tables.
    [012] Where processing begins depends on the type of fixed format document 106 being parsed. A native fixed-format document 106a created directly from a stream-format source document contains some or all of the basic physical layout elements. Generally speaking, data extracted from a document in fixed native format. The built-in data structures are extracted by the parser and are available for immediate use by the document converter; however, in some cases, secondary reformatting or another secondary processor is applied to organize or standardize the data. In contrast, all information in a 106b image-based fixed-format document created by digitally imaging a physical document (eg, by scanning or photographing) is stored as a series of page images with no additional data (ie, no text run or path). In this case, the optional optical character recognition engine 202 analyzes each page image and creates corresponding physical layout objects. Once the physical layout objects 208 are available, the layout analysis engine 204 analyzes the document layout in fixed format. After layout analysis is complete, the semantic analysis engine 206 enriches the logical layout objects with semantic information obtained from analyzing the physical layout objects and/or logical layout objects.
    [013] Figure 3 is a flowchart showing an embodiment of the formula detection method 300 performed by the formula detection engine 100. The formula detection engine 100 creates in 302 the initial formula areas by identifying elements in the layout objects that are potentially part of a mathematical formula and group these elements into formula areas based on the relative position of those elements. The formula detection engine 100 then eliminates 304 overlap between formula areas and any surrounding normal text (that is, text elements that are not part of a mathematical formula) by subdividing formula areas by overlapping normal text and splitting the formula areas based on horizontal spacing. Finally, the formula detection engine 100 reconstructs into 306 the mathematical formulas as capable elements for flow by merging any neighboring formula areas based on proximity.
    [014] Figure 4 is a flowchart showing an modality of the process of creating in 302 the initial formula areas used in the formula detection method 300. The formula detection mechanism 100 starts by identifying in 402 formula seeds in the analyzed elements syntactically. More generally, a formula seed is a text element that carries some indication of being part of a formula. To identify formula seeds, the formula detection engine 100 looks for one or more of the following, without limitation: plain text recorded with fonts used exclusively, or almost exclusively, to display mathematical expressions and mathematical operators, symbols, or keywords , which are used exclusively, or almost exclusively, in mathematical formulas.
    [015] For example, a word processor can represent formulas with a mathematical font such as Microsoft Corporation's Cambria® Math, while a document preparation system (eg LaTeX) can use multiple mathematical font families, such as like Computer Modern math fonts. The formula detection engine 100 also considers the presence of math operators, symbols, and keywords to identify formula seeds because some document processors do not use any special math fonts for math. Examples of mathematical operators, symbols or keywords used for formula detection include operators such as "∑" and "+", symbols such as "π", and keywords such as "cos" representing the cosine function. Generally speaking, the formula detection engine 100 uses keywords that have no meaning in normal language (eg, "tan"). In various embodiments, fonts used to identify formula seeds include, but are not limited to, some or all of the following fonts: Cambria® Math, Computer Modern math Italic (cmmi), Computer Modern math bold Italic (cmmib), Computer Modern math extension (cmex), Computer Modern math symbols (cmsy), Computer Modern bold math symbols (cmsy), American Math Society extra math symbols - first series (masm), American Math Society extra math symbols - second series (msbm), the extended set of integrals for Computer Modern (esint), MathTime TeX math italic (mtmi), MathTime TeX math symbols (mtsy), MathTime TeX math extenion (mtex), and Roland Waldi's symbols (wasy). In some embodiments, formula seeds include, but are not limited to, some or all Unicode characters or character sets in the ranges #2200-#22FF (mathematical operators), #27C0-#27EF (miscellaneous math symbols -A), #2980-#29FF (mixed math symbols-B), and #2A00-#2AFF (additional math operators) as formula seeds. In various embodiments, formula seeds include, but are not limited to, some or all of the following textual keywords: det, sen, cos, tg, tan, ctg, ctan, senh, cosh, tanh, ctanh, log , in, gcd, arcsen, arcs, arctan, sec, csc, max, min, inf, sup, lim, sgn, exp, mod and var.
    [016] Once the formula seeds are identified, the formula detection engine 100 defines at 404 a formula area (i.e., an outline) around each of the detected formula seeds. The formula detection engine 100 then expands the contours around the formula seeds by 406 to create the initial formula areas. In one modality, it adds all page elements in the vicinity of a formula seed that have mathematical element properties, until there are no such elements remaining. To decide whether or not a page element should be included in the formula area, the formula detection engine 100 looks for properties identifying the page element as a potential mathematical formula element. In various embodiments, formula detection engine 100 considers properties including, but not limited to, Euclidean distance from formula area to page element, page element text font, presence of mathematical operators, symbols and/or numeric characters (ie digits) in the page element, and the dimensions of the page element (ie whether it is taller or wider than standard text elements). All page elements enclosed by the formula area confinement box are assumed to be the captured elements.
    [017] Figure 5A graphically illustrates a selected part of the 500 parsed data from a fixed format document with examples of the 502a-502d formula seeds identified by the formula 100 detection mechanism. The formula seeds identified by the 502a font are enclosed with dashed-line rectangles, and formula seeds identified using 502b math operators, 502c symbols, and 502d keywords are enclosed with solid-line ovals.
    [018] Figure 5B graphically illustrates the initial formula area 504 created by the formula detection engine 100 outlined by a confinement box. In the illustrated modality, the result is a formula area bounded by rectangles that contain one or more mathematical formulas. Where mathematical formulas are sufficiently separated from each other, each formula area will capture a single mathematical formula; however, in cases where multiple mathematical formulas are in close proximity, separate mathematical formulas can be captured in a single formula area. Also, when more than one formula is captured in a single formula area, the expanded formula area may overlap the surrounding normal text. In the illustrated mode, the expanded formula area has captured multiple 506 math formulas and overlays the 508 normal text surrounding the math formulas.
    [019] Figure 6 is a flowchart showing an embodiment of the process of eliminating in 304 overlap between formula areas and normal text used in formula detection method 300. The formula detection mechanism 100 starts by analyzing at 600 each area of formula to determine whether or not the formula area overlaps any normal text areas. If the formula area overlaps any normal text, the formula detection engine 100 continues to group each captured element into 602 based on the captured element's vertical position. A new formula area is created at 604 around each new grouping of captured elements. The formula detection engine 100 analyzes the contents of the new formula areas for overlay with normal text, and any new formula areas containing normal text are further split. Analyzing, splitting, and iteratively collapsing each new formula area, as needed, allows the formula detection engine 100 to handle cases where a formula area overlaps multiple lines of text. Since the formula areas are grouped vertically, the formula detection engine 100 continues to divide each formula area into 606 according to horizontal spacing between the captured elements.
    [020] Figures 7A-7D graphically illustrate the sub-process of eliminating in 304 overlap between formula areas and normal text used in formula detection method 300 applied to initial formula area 504. Figure 7A shows the initial formula area 504 divided by the two horizontal dividers 700a, 702a used to group captured elements from the initial formula area 504 based on vertical position. The horizontal lines 700a, 702a correspond to the height of a line of normal text. The first horizontal line 700a corresponds to the top edge (ie, boundary) of the box enclosing the normal text line containing the mathematical formula. The second horizontal line 702a corresponds to the bottom edge of the box enclosing the normal text line containing the math formulas.
    [021] Dividers 700a, 702a divide the initial formula area 504 into an upper region 704a, a central region 706a, and a lower region 708a, which correspond to the areas above, in-line, and below the normal text line. If one of the primary dividers 700a, 702a intersects with captured elements, the formula detection mechanism 100 establishes an upper secondary horizontal divider 710a and a secondary lower horizontal divider 712a, as necessary, above and below the highest and most positions. low cross-captured elements. It should be appreciated by those skilled in the art that the secondary dividers 710a, 712a need not be established if the primary dividers 700a, 702a do not intersect with any captured elements. Alternatively, secondary dividers 710a, 712a may be collinear with primary dividers 700a, 702a.
    [022] Only those captured elements which lie completely above the highest dividing line 700a, 710a established by the detection mechanism of formula 100 are placed in a first group corresponding to the upper region 704a. In the illustrated mode, only the "n" and "*" symbols are within the first group. Similarly, only those captured elements which lie completely below the lowermost dividing line 702a, 712a established by the detection mechanism of formula 100 are placed in a second group corresponding to the lower region 708a. In the illustrated modality, the terms "k=0" and "n=1" and the text, equations and formulas in the line below these terms become part of the second group. All captured elements that are not in the first group or the second group are placed in a third group corresponding to central region 706a. In other words, the third group contains the captured elements that lie completely between the primary dividers 700a, 702a, are crossed by one or both of the primary dividers 700a, 702a, or vertically overlap a crossed captured element. Any empty groups are discarded.
    [023] Figure 7B illustrates the operation of creating in 604 new formula areas around the groups based on the vertical position of the captured elements. Up to three new formula areas are created from the initial formula area 504. The formula detection engine 100 optionally reduces the dimensions of each new formula area to a suitable extent sufficient to enclose all formula elements captured in it. In the illustrated embodiment, initial formula area 504 has been subdivided into the three new formula areas 714, 716, and 718 and initial formula area 500 has been discarded. Each of the new formula areas 714, 716 and 718 undergoes the overlap identification operation at 600. In the illustrated embodiment, the two new higher formula areas 714, 716 do not require any additional vertical division. The third new formula area 718 still contains vertical overlap between formula areas and normal text. In this way, the formula detection engine 100 repeats the grouping by means of the vertical position operation at 602 and the new formula area creation operation at 604 at the third new formula area 718. The lower dividing line 702b intersects with the sum operator, and the upper dividing line 700b intersects with both the square root operator and the sum operator. It should be noted that the upper secondary divisor 710b is set above the highest cross captured element (ie, the square root operator). As before, dividers 700b, 702b divide area of formula 718 into upper region 704b, central region 706b, and lower region 708b. Formula detection engine 100 regroups the captured elements in formula region 718 in the manner described above.
    [024] Figure 7C illustrates the final result of the iterative application of the overlap identification operation at 600, the grouping through the vertical position operation at 602 and the new formula area creation operation at 604. The third new region of formula 718 has been divided into the two new areas of formulas 724, 726. The first area of formula 724 corresponds to the upper region 704b of the region of formula 718, and the second area of formula 726 corresponds to the central region 706b of the region of formula 718. The empty group corresponding to central region 706b of the formula region 718 was discarded. None of the remaining formula areas 724, 726 require any additional vertical grouping.
    [025] Figure 7D illustrates the operation of dividing into 606 formula areas based on the horizontal spacing between captured elements. The formula detection engine 100 starts by horizontally scanning each formula area and determining the horizontal distance between each pair of consecutive captured elements in a formula area. In some embodiments, the horizontal scan direction corresponds to the document language reading direction. In other embodiments, the horizontal sweep direction is left-to-right or right-to-left regardless of the document language. The formula detection engine 100 splits the formula area between two consecutive captured elements separated by a horizontal distance greater than a selected threshold distance to create new formula areas. In many modalities, the horizontal distance between consecutive captured elements is determined by the white space. The 730 dotted lines shown in Figure 7D indicate where the horizontal separation exceeds the selected threshold. In some modalities, a single threshold is used. In other modalities, the threshold varies based on the surrounding text.
    [026] Figure 8 is a flowchart showing a modality of the sub-process of reconstructing in 306 the individual formulas by grouping the formula boxes. The 100 formula detection engine starts by finding at 800 the normal text elements appearing to the left and right of each formula area. The formula detection engine 100 uses information about the positions of neighboring text elements to prevent merging of formula areas appearing on different lines on a page. The formula detection engine 100 then generates sets of fusion candidates from the available formula areas. Areas of formulas that overlap horizontally (i.e., are at least partially vertically aligned) and are within a selected vertical separation distance are grouped into 802 as merge candidates. The formula detection engine sorts (for example, sorts) into 804 the merge candidates according to the vertical distance between the formula areas. In some modalities, the merger candidates are ranked in ascending order. In other modalities, the merger candidates are ranked in descending order. Working with the merge candidates ranked first to last, the formula detection engine 100 replaces in 806 two formula areas constituting the merge candidate with a new formula area, provided the two formula areas have not already been merged in a single formula area and their vertical positions are not associated with the vertical positions of separate lines. In some embodiments, whether or not formula areas are on the same line on the page is determined from the normal text to the right and/or left of each formula area. If the normal text to the right or left of each formula area is the same, the formula areas are determined to be located within the same line on the page. Conversely, a difference in normal text to the right or left of the formula areas indicates that the formula areas are located on different lines on the page. The contour of the merged formula area is defined by the maximum extents of the two formula areas constituting the merge candidate. In other words, the new formula area is defined by selecting the highest, lowest, leftmost, and rightmost contour of the top, bottom, left, and right contours of the merge candidates.
    [027] Figures 9A and 9B graphically illustrate selected operations of the subprocess of reconstructing into 306 the individual formulas applied to the running text 500. Figure 9A shows areas of selected formulas grouped into merge candidates. The first group of 900a fusion candidates includes formula areas 914a, 916a and 924a. The second group of 900b fusion candidates includes formula areas 914b, 916b and 924b. Formula areas 924a and 924b are not candidates for merging with formula areas 926a and 926b, respectively, because the vertical separation distance exceeds the selected threshold. Additionally, formula areas 926a and 926b are not fused with fusion groups 900a and 900b, respectively, because the left and right normal text differs between them (i.e., the formula areas are on different lines in the page). Figure 9B shows the final solution with each of the four formulas 902a-902d properly separated from each other and from the surrounding normal text.
    [028] As used in this document, the terms "area", "contour", "box" are used interchangeably. Similarly, the terms "line" and "divisor" are used interchangeably. It should be understood by those skilled in the art that the contours and dividers described in this document need not actually be represented and/or displayed visually during the formula detection method. Additionally, contours and dividers are not limited to boxes or lines. Contours and dividers may have other shapes (eg curves) without departing from the scope and spirit of the present invention. Also, contours and dividers can simply use coordinates or other reference systems. Terms indicating shapes (eg, rectangle, box, line and oval) should not be construed as limiting and should be read broadly to encompass any suitable outline or divider, as appropriate, unless the descriptive report expressly states otherwise .
    [029] The formula detection engine and associated formula detection method described in this document are useful for identifying each distinct mathematical formula appearing in a fixed format document and for converting each identified mathematical formula to an element in stream format. In various embodiments, output from the formula detection engine is further processed by additional formatting engines within the document processor before being serialized.
    [030] Although the invention has been described in the general context of program modules that run in combination with an application program that runs in an operating system on a computer, those skilled in the art will recognize that the invention can also be implemented in combination with other program modules. Generally speaking, program modules include routines, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types.
    [031] The modalities and functionalities described in this document may operate through a multitude of computing systems including, without limitation, desktop computer systems, wired and wireless computing systems, mobile computing systems (eg, telephones furniture, netbooks, tablet or whiteboard computers, notebooks and laptops), handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, and mainframe computers. Figure 10 illustrates an exemplary computing device 1000 running an embodiment of the formula detection engine 100. Furthermore, the modalities and functionality described in this document can operate in distributed systems (eg, cloud-based computing systems) where application functionality, memory, data storage and retrieval, and various processing functions can be operated remotely from each other via a distributed computing network, such as the Internet or an intranet. User interfaces and information of various types can be displayed via displays built into computing devices or via remote display units associated with one or more computing devices. For example, user interfaces and information of various types can be displayed and interacted with on a wall surface onto which user interfaces and information of various types are projected. Interaction with the multitude of computing systems with which embodiments of the invention can be practiced include keystroke input, touch screen input, voice or other audio input, gesture input where an associated computing device is equipped with detection functionality (eg camera) to capture and interpret user gestures to control computing device functionality and more. Figures 11 to 13 and associated descriptions provide a discussion of a variety of operating environments in which embodiments of the invention may be practiced. However, the devices and systems illustrated and discussed in connection with Figures 11 to 13 are for purposes of example and illustration and are not limiting of the vast number of computing device configurations that can be used to practice embodiments of the invention described in this document. .
    [032] Figure 11 is a block diagram illustrating physical components (i.e., hardware) of a computing device 1100 with which embodiments of the invention can be practiced. The computing device components described below may be suitable for the computing devices described above. In a basic configuration, computing device 1100 may include at least one processing unit 1102 and system memory 1104. Depending on the configuration and type of computing device, system memory 1104 may comprise, but is not limited to, that is, volatile storage (eg, random access memory), non-volatile storage (eg, read-only memory), flash memory, or any combination of such memories. System memory 1104 may include an operating system 1105 and one or more program modules 1106 suitable for running software applications 1120 such as formula detection engine 100, parser 110, document converter 112, and serializer 114. Operating system 1105, for example, may be suitable for controlling the operation of computing device 1100. Furthermore, embodiments of the invention may be practiced in combination with a graphics library, other operating systems, or any other application program. and is not limited to any particular application or system. This basic configuration is illustrated in Figure 11 by these components within a dashed line 1108. Computing device 1100 may have additional features or functionality. For example, computing device 1100 may also include additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated in Figure 11 by means of a removable storage device 1109 and a non-removable storage device 1110.
    [033] As stated above, various program modules and data files may be stored in system memory 1104. While running in processing unit 1102, program modules 1106, such as formula detection engine 100, parser 110, document processor 112 and serializer 114 can perform processes including, for example, one or more of the stages of formula detection method 300. The above-mentioned process is an example, and processing unit 1102 can run other processes. Other program modules that can be used in accordance with embodiments of the present invention may include e-mail and contact applications, word processing applications, spreadsheet applications, database applications, slide show applications, computer programs. drawing or computer-aided applications, etc.
    [034] In addition, modalities of the invention can be practiced in an electrical circuit comprising distinct electronic elements, compressed or integrated electronic chips containing logic gates, a circuit using a microprocessor, or in a single chip containing electronic elements or microprocessors. For example, embodiments of the invention can be practiced through a system on a chip (SOC) where each or many of the components illustrated in Figure 11 can be integrated into a single integrated circuit. Such a SOC device can include one or more processing units, graphics units, communications units, system virtualization units and various application functionalities, all of which are integrated (or "burned") into the chip substrate as a single integrated circuit. When operating through a SOC, the functionality described in this document with respect to the formula detection mechanism 100, the parser 110, the document processor 112 and the serializer 114 can be operated via application specific logic integrated with others 1100 computing device components on the single integrated circuit (chip). Embodiments of the invention may also be practiced using other technologies capable of performing logical operations such as, for example, AND, OR and NOT, including, but not limited to, mechanical, optical, fluidic and quantum technologies. Furthermore, embodiments of the invention can be practiced on a general purpose computer or on any other circuits or systems.
    [035] The computing device 1100 may also have one or more input devices 1112 such as a keyboard, a mouse, a pen, a sound input device, a touch input device, etc. The 1114 output device(s) such as a display, speakers, a printer, etc. can also be included. The devices mentioned above are examples and others can be used. Computing device 1100 may include one or more communication connections 1116 allowing communications with other computing devices 1118. Examples of suitable communication connections 1116 include, but are not limited to, RF transmitter, receiver and/or circuitry. transceiver; universal serial bus (USB), parallel or serial ports, and other connections suitable for use with applicable computer-readable media.
    [036] Modalities of the invention, for example, can be implemented as a computer process (method), a computer system, or as an article of manufacture, such as a computer program product or computer readable media. The computer program product may be computer storage media readable by a computer system and encoding a computer program of instructions for executing a computer process.
    [037] The term computer readable media as used in this document may include computer storage media and communication media. Computer storage media may include volatile and non-volatile removable and non-removable media implemented in any method or technology for storing information, such as computer readable instructions, data structures, program modules or other data. System memory 1104, removable storage device 1109, and non-removable storage device 1110 are all examples of computer storage media (i.e., memory storage). Computer storage media may include, but are not limited to, RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other storage optical, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other media that can be used to store information and that can be accessed by the computing device 1100. Any such computer storage media may be part of the 1100 computing device.
    [038] Communication media may be incorporated by computer-readable instructions, data structures, program modules or other data into a modulated data signal, such as a carrier wave or other transport mechanism, and include any media for the delivery of information. The term "modulated data signal" can describe a signal that has one or more characteristics set or changed in a manner such as to encode information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct wired connection, and wireless media such as acoustic, radio frequency (RF), infrared and other wireless media.
    [039] Figures 12A and 12B illustrate a mobile computing device 1200, for example, a mobile phone, a smart phone, a personal computer, a laptop and the like, with which embodiments of the invention can be practiced. Referring to Figure 12A, an exemplary mobile computing device 1200 for implementing the modalities is illustrated. In a basic configuration, mobile computing device 1200 is a handheld computer having both input elements and output elements. Mobile computing device 1200 typically includes a display 1205 and one or more input keys 1210 that allow the user to input information into mobile computing device 1200. Display 1205 of mobile computing device 1200 may also function as an input device ( for example, a touchscreen display). If included, an optional 1215 side input element allows additional user input. Side input element 1215 can be a rotary switch, a knob, or any other type of manual input element. In alternative embodiments, mobile computing device 1200 may incorporate more or less input elements. For example, the 1205 display may not be a touchscreen in some modalities. Also in another alternative embodiment, mobile computing device 1200 is a portable telephone system, such as a cell phone. The 1200 mobile computing device can also include an optional 1235 mini keyboard. The optional 1235 mini keyboard can be either a physical mini keyboard or a “soft” mini keyboard generated on the touchscreen display. In various embodiments, the output elements include a 1205 display for showing a graphical user interface (GUI), a 1220 visual indicator (e.g., a light-emitting diode) and/or an audio transducer 1225 (e.g., a speaker). In some embodiments, mobile computing device 1200 incorporates a vibration transducer to provide the user with tactile feedback. Also in another embodiment, the mobile computing device 1200 incorporates input and/or output ports such as an audio input (eg a microphone jack), an audio output (eg a phone jack head), and a video output (eg an HDMI port) for sending signals to or receiving signals from an external device.
    [040] Figure 12B is a block diagram illustrating the architecture of a modality of a mobile computing device. That is, mobile computing device 1200 may incorporate a system (i.e., architecture) 1202 to implement some modalities. In one embodiment, system 1202 is implemented as a "smart phone" capable of running one or more applications (e.g., browser, email, scheduling, contact managers, messaging clients, games, and media clients/players). In some embodiments, system 1202 is integrated as a computing device, such as an integrated personal digital assistant (PDA) and cordless telephone.
    [041] One or more 1266 application programs may be loaded into memory 1262 and run by operating system 1264 or in association with it. Examples of application programs include telephone dialer programs, e-mail programs, personal information management (PIM) programs, word processing programs, spreadsheet programs, Internet browser programs, messaging programs, and so on. System 1202 also includes a non-volatile storage area 1268 within memory 1262. The non-volatile storage area 1268 can be used to store permanent information that should not be lost if system 1202 is turned off. Application programs 1266 may use and store information in non-volatile storage area 1268, such as e-mail or other messages used by an e-mail application, and the like. A synchronization application (not shown) also resides in system 1202 and is programmed to interact with a corresponding synchronization application residing on a host computer to keep information stored in nonvolatile storage area 1268 synchronized with corresponding information stored in the host computer. As should be understood, other applications can be loaded into memory 1262 and run on mobile computing device 1200, including formula detection engine 100, parser 110, document processor 112, and serializer 114 described herein.
    [042] System 1202 has a 1270 power supply, which can be implemented as one or more batteries. Power supply 1270 may additionally include an external power source, such as an AC adapter or a powered docking cradle that supplements or recharges the batteries.
    [043] System 1202 may also include a radio 1272 that performs the function of transmitting and receiving radio frequency communications. The 1272 radio promotes wireless connectivity between the 1202 system and the "outside world" through a communications carrier or service provider. Transmissions to and from radio 1272 are conducted under the control of operating system 1264. In other words, communications received by radio 1272 can be disseminated to application programs 1266 via operating system 1264, and vice versa.
    [044] The 1272 radio allows the 1202 system to communicate with other computing devices, such as over a network. Radio 1272 is an example of communication media. Communication media typically can be incorporated by computer readable instructions, data structures, program modules, or other data into a modulated data signal, such as a carrier wave or other transport mechanism, and include any information delivery media. . The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in a manner such as to encode information in the signal. By way of example, and not limitation, communication media include wired media such as a wired network or direct wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. The term computer readable media as used in this document includes both storage media and communication media.
    [045] This system modality 1202 provides notifications using visual indicator 1220 which can be used to provide visual notifications and/or an audio interface 1274 producing audible notifications via audio transducer 1225. In the illustrated embodiment, visual indicator 1220 is a light emitting diode (LED) and the 1225 audio transducer is a speaker. These devices can be coupled directly to the 1270 power supply in such a way that, when activated, they remain on for a duration of time by the notification mechanism even though the 1260 processor and other components may be turned off to save battery power. The LED can be programmed to remain on indefinitely until the user takes action to indicate the device's powered on status. The 1274 audio interface is used to provide audible signals to the user and to receive audible signals from the user. For example, in addition to being coupled to audio transducer 1225, audio interface 1274 can also be coupled to a microphone to receive audible input, such as to facilitate a telephone conversation. In accordance with embodiments of the present invention, the microphone can also serve as an audio sensor to facilitate notification control, as will be described below. System 1202 can additionally include a video interface 1276 that enables an operation of a built-in camera 1230 to record still images, video stream, and the like.
    [046] A mobile computing device 1200 implementing system 1202 may have additional features or functionality. For example, mobile computing device 1200 may also include additional data storage devices (removable and/or non-removable) such as magnetic disks, optical disks or tape. Such additional storage is illustrated in Figure 12B by non-volatile storage area 1268. Computer storage media may include volatile and non-volatile removable and non-removable media implemented in any method or technology for storing information, such as computer readable instructions, data structures, program modules, or other data.
    [047] Data/information generated or captured by mobile computing device 1200 and stored via system 1202 may be stored locally on mobile computing device 1200, as described above, or the data may be stored on any number of storage media that can be accessed by radio device 1272 or via a wired connection between mobile computing device 1200 and a separate computing device associated with mobile computing device 1200, e.g., a server computer in a distributed computing network , just like the Internet. As will be appreciated such data/information may be accessed via mobile computing device 1200 via radio 1272 or via a distributed computing network. Similarly, such data/information can be readily transferred between computing devices for storage and use in accordance with well-known data/information transfer and storage devices, including email and collaborative data/information sharing systems.
    [048] Figure 13 illustrates an architectural embodiment of a system for providing the formula detection mechanism 100, the parser 110, the document processor 112 and the serializer 114 to one or more client devices, as described above. Content developed, interacted with or edoed in association with formula detection engine 100, parser 110, document processor 112 and serializer 114 can be stored in different communication channels or in other types of storage. For example, multiple documents can be stored using a directory service 1322, a web portal 1324, a mailbox service 1326, an instant messaging store 1328, or a social networking site 1330. The formula detection mechanism 100 , parser 110, document processor 112, and serializer 114 can use any of these types of systems or the like to enable data usage as described in this document. A server 1320 can provide formula detection engine 100, parser 110, document processor 112, and serializer 114 to clients. As an example, server 1320 may be a network server providing formula detection engine 100, parser 110, document processor 112, and serializer 114 over the network. Server 1320 may provide formula detection mechanism 100, parser 110, document processor 112, and serializer 114 over the network to clients via a network 1315. By way of example, client computing device 1318 may be implemented as the computing device 1100 and incorporated in a personal computer 1318a, a tablet 1318b and/or a mobile computing device 1318c (e.g., a smart phone). Any of these embodiments of client computing device 1318 can retrieve content from storage 1316.
    [049] Modalities of the present invention, for example, have been described above with reference to block diagrams and/or operational illustrations of methods, systems and computer program products according to the embodiments of the invention. Functions/procedures noted in blocks may occur out of order as shown in any flowchart. For example, two blocks shown in succession may actually be executed substantially concurrently or the blocks may sometimes be executed in reverse order, depending on the functionality/procedures involved.
    [050] Although certain embodiments of the invention have been described, other embodiments may exist. Furthermore, although embodiments of the present invention have been described as being associated with data stored in memory and other storage media, data may also be stored or read on other types of computer readable media, such as secondary storage devices, or be it hard disks, floppy disks or a CD-ROM, an Internet carrier wave, or other forms of RAM or ROM. Additionally, the stages of the disclosed methods can be modified in any way, including reordering stages and/or inserting or deleting stages, without departing from the invention.
    [051] In various embodiments, the types of networks used for communication between the computing devices that make up the present invention include, but are not limited to, the Internet, an intranet, wide area networks (WAN), area networks local area (LAN) and virtual private networks (VPN). In the present application, networks include the corporate network and the network through which the client computing device accesses the corporate network (ie, the customer network). In one modality, the customer network is part of the company network. In another embodiment, the client network is a separate network accessing the corporate network through externally available entry points, such as a gateway, a remote access protocol, or a public or private Internet address.
    [052] The description and illustration of one or more embodiments provided in this application are not intended to limit or restrict in any way the scope of the invention as claimed. The modalities, examples and details provided in this application are considered sufficient to convey ownership and enable others to make and use the best mode of the claimed invention. The claimed invention is not to be construed as being limited to any embodiment, example, or detail provided in this application. Regardless of whether shown and described in combination or separately, the various features (both structural and methodological) are intended to be selectively included or omitted to produce a modality with a particular set of features. Having been provided with the description and illustration of the present application, those skilled in the art may view variations, modifications and alternative modalities being included in the spirit of the broader aspects of the claimed invention and in the general inventive concept incorporated in this application which does not depart from the scope further broad.
    权利要求:
    Claims (15)
    [0001]
    1. Formula detection method to identify a mathematical formula contained in parsed data from a fixed-format document and allow the mathematical formula to be transformed into a stream-format element, the method characterized by the fact that it comprises the steps of : identify (402) a formula seed indicative of a mathematical formula in parsed data from a fixed-format document, where a formula seed is a text element that bears some indication of being part of a formula; create (404) a formula area that encloses the identified formula seed; expand (406) the formula area (504) to include formula elements next to the identified formula seed, where a formula element is a page element that is potentially part of a formula, where the expanded formula area includes a plurality of captured elements comprising the formula seed and the formula elements included in the expanded formula area; placing (602) the plurality of captured elements included in the expanded formula area into groups based on the vertical position of the plurality of captured elements relative to a line of normal text that overlaps the expanded formula area; create (604) a new formula area for each of the groups; dividing (606) each new formula area into divided formula areas based on the horizontal separation between consecutive captured elements of the plurality of captured elements of the respective new formula area; generating (802) a set of split formula areas from the split formula areas, wherein the split formula areas within the generated set at least partially horizontally overlap; and merging (806) the areas of the divided formula into the generated set that appear on the same line on a page into a single formula area.
    [0002]
    2. Formula detection method, according to claim 1, characterized in that the step of identifying a formula seed comprises the step of identifying text rendered in a mathematical font.
    [0003]
    3. Formula detection method, according to claim 1 or 2, characterized in that the step of identifying a formula seed comprises the step of identifying selected mathematical symbols, selected mathematical operators and selected mathematical keywords.
    [0004]
    4. Formula detection method according to any one of claims 1 to 3, characterized in that the step of placing the plurality of captured elements included in the expanded formula area into groups based on the vertical position of the plurality of elements captured relative to a line of normal text that overlaps the expanded formula area further comprises the steps of: placing captured elements from the plurality of captured elements that appear above an upper edge of the normal text line overlapping the expanded formula area in a first group; place captured elements from the plurality of captured elements that appear below a lower border of the normal text line by overlaying the expanded formula area in a second group; and placing each remaining captured element of the plurality of captured elements into a third group.
    [0005]
    5. Formula detection method according to any one of claims 1 to 3, characterized in that the step of placing the plurality of captured elements included in the expanded formula area into groups based on the vertical position of the plurality of elements captured in relation to a normal text line that overlaps the expanded formula area further comprises the steps of: placing captured elements from the plurality of captured elements that appear above a captured element intercepted by a top edge of a normal text line in a first group; placing captured elements from the plurality of captured elements that appear below a captured element intercepted by a lower edge of a normal text line into a second group; and placing each remaining captured element of the plurality of captured elements into a third group.
    [0006]
    6. Formula detection method according to claim 1, characterized in that the step of generating a set of split formula areas from the split formula areas further comprises the step of placing split formula areas on the set generated when a vertical separation distance between two split formula areas does not exceed a selected distance threshold; and wherein the step of generating a set of split formula areas forms the split formula areas further comprises the step of arranging the split formula areas in the generated set by vertical position.
    [0007]
    7. Formula detection method according to any one of claims 1 to 6, characterized in that the step of dividing each new formula area into divided formula areas based on the horizontal separation between consecutive captured elements of the plurality of captured elements of the respective new formula area is based on a selected distance threshold, where the selected distance threshold varies based on the consecutive captured elements of the formula being evaluated.
    [0008]
    8. Formula detection method, according to any one of claims 1 to 7, characterized in that the step of merging the split formula areas further comprises the steps of: identifying normal text appearing on the left and right of each divided formula area; and determining that two areas of the split formula appear on the same line on a page when the normal text appearing to the left and right of each of the two areas of the split formula is identical
    [0009]
    9. System for detecting a formula appearing in a fixed-format document, characterized in that said system characterized by comprising an application of a formula detection mechanism operable to: identify (402) a formula seed indicative of a mathematical formula on parsed data from a fixed-format document, where a formula seed is a text element that carries some indication of being part of a formula; create (404) a formula area enclosing the identified formula seed; expand (406) the formula area (504) to include formula elements next to the identified formula seed, where a formula element is a page element that is potentially part of a formula, where the expanded formula area includes a plurality of captured elements comprising the formula seed and the formula elements included in the expanded formula area; place (602) the plurality of captured elements included in the expanded formula area into groups based on the vertical position of the captured plurality of elements relative to a line of normal text that overlaps the expanded formula area; create (604) a new formula area around each of the groups; dividing (606) each new formula area into divided formula areas based on the horizontal separation between consecutive captured elements of the plurality of captured elements of the respective formula area; generating (802) a set of split formula areas from the split formula areas, wherein the split formula areas in the at least horizontally generated set overlap horizontally; and merging (806) the areas of the divided formula into the generated set that appear on the same line on a page into a single formula area.
    [0010]
    10. System according to claim 9, characterized in that the application of formula detection mechanism is operable to identify plain text rendered in a mathematical font as formula seeds; or where the formula detection engine application is operable to identify selected math symbols, selected math operators, and selected math keywords as formula seeds.
    [0011]
    11. System according to claim 9 or 10, characterized in that the application of formula detection mechanism is operable to: place formula elements appearing above an upper limit of a normal text line in a first group; place formula elements appearing below a lower boundary of a normal text line in a second group; and place each remaining formula element into a third group.
    [0012]
    12. System according to any one of claims 9 to 10, characterized in that the application of a formula detection mechanism is operable to: place the captured elements of the plurality of captured elements that appear above a cross formula element by a top edge of a line of normal text in a first group; place the captured elements from the plurality of captured elements that appear below a formula element crossed by a lower edge of a normal text line into a second group; and placing each remaining captured element of the plurality of captured elements into a third group.
    [0013]
    13. System, according to any one of claims 9 to 12, characterized in that the application of formula detection mechanism is operable to: identify a normal text appearing on the left and right of the divided formula area; and determining that the two split formula areas appear on the same line on a page when the normal text appearing to the left and right of two split formula areas is identical.
    [0014]
    14. Computer readable media having a method for identifying a mathematical formula contained in the analyzed data from a fixed format document and allowing the mathematical formula to be transformed into a stream format element, the method characterized by the fact that it comprises the steps of: identifying (402) a formula seed indicative of a mathematical formula in parsed data from a fixed-format document, where a formula seed is a text element that carries some indication of being part of a formula ; create (404) a formula area that encloses the identified formula seed; expand (406) the formula area (504) to include formula elements next to the identified formula seed, where a formula element is a page element that is potentially part of a formula, where the expanded formula area includes a plurality of captured elements comprising the formula seed and the formula elements included in the expanded formula area; place captured elements from the plurality of captured elements that appear above a formula element crossed by a top edge of a normal text line overlaying the expanded formula area in a first group; place captured elements from the plurality of captured elements that appear below a formula element crossed by a lower edge of the normal text line overlaying the expanded formula area into a second group; placing each remaining captured element of the plurality of captured elements into a third group; creating (604) a new formula area for each of said first group, said second group, and said third group; dividing (606) each new formula area into divided formula areas based on the horizontal separation between consecutive captured elements of the plurality of captured elements of the respective new formula area; generating (802) a set of split formula areas from the split formula areas, wherein the split formula areas within the generated set at least partially horizontally overlap; identify normal text appearing to the left and right of each area of the split formula; determine that two split formula areas appear on the same line on a page when the normal text that appears to the left and right of each of the two formula areas is identical; and merging (806) the formula areas divided in the generated set that appear on the same line on a page into a single formula area.
    [0015]
    15. Computer-readable media according to claim 14, characterized in that the step of identifying a formula seed comprises the step of identifying text rendered in a mathematical font; or wherein the step of identifying a formula seed comprises the step of identifying selected mathematical symbols, selected mathematical operators, and selected mathematical keywords; or wherein the step of generating a set of split formula areas further comprises the step of arranging the split formula areas within the set generated by vertical position.
    类似技术:
    公开号 | 公开日 | 专利标题
    BR112014017832B1|2021-07-06|formula detection method for identifying a mathematical formula, system for detecting a formula that appears in a fixed-format document and computer-readable media
    KR101872564B1|2018-06-28|Borderless table detection engine
    TWI595366B|2017-08-11|Detection and reconstruction of east asian layout features in a fixed format document
    US9965444B2|2018-05-08|Vector graphics classification engine
    US20130191732A1|2013-07-25|Fixed Format Document Conversion Engine
    US10025979B2|2018-07-17|Paragraph property detection and style reconstruction engine
    US20140258852A1|2014-09-11|Detection and Reconstruction of Right-to-Left Text Direction, Ligatures and Diacritics in a Fixed Format Document
    同族专利:
    公开号 | 公开日
    KR20140116428A|2014-10-02|
    EP2807603A1|2014-12-03|
    BR112014017832A2|2017-06-20|
    CA2863522A1|2013-08-01|
    EP2807603B1|2020-03-18|
    BR112014017832A8|2021-03-02|
    KR101812380B1|2017-12-26|
    RU2585972C2|2016-06-10|
    US9928225B2|2018-03-27|
    AU2012367116A1|2014-08-07|
    CN104067292B|2017-05-03|
    AU2012367116B2|2017-10-19|
    RU2014130243A|2016-02-10|
    JP5974115B2|2016-08-23|
    MX2014008560A|2014-09-26|
    CN104067292A|2014-09-24|
    JP2015505113A|2015-02-16|
    CA2863522C|2018-08-28|
    US20130205200A1|2013-08-08|
    WO2013110285A1|2013-08-01|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    JPH0253821B2|1985-09-06|1990-11-19|Fujitsu Ltd|
    US5212769A|1989-02-23|1993-05-18|Pontech, Inc.|Method and apparatus for encoding and decoding chinese characters|
    EP0667567B1|1993-12-30|2001-10-17|Xerox Corporation|Apparatus and method for supporting the implicit structure of freeform lists, outlines, text, tables, and diagrams in a gesture-based input system and editing system|
    US6370269B1|1997-01-21|2002-04-09|International Business Machines Corporation|Optical character recognition of handwritten or cursive text in multiple languages|
    JPH10224789A|1997-02-07|1998-08-21|Matsushita Electric Ind Co Ltd|Image data processor and its method|
    JPH11259477A|1998-03-13|1999-09-24|Toshiba Corp|Document processing system and storage medium|
    US6081381A|1998-10-26|2000-06-27|Polametrics, Inc.|Apparatus and method for reducing spatial coherence and for improving uniformity of a light beam emitted from a coherent light source|
    US6757870B1|2000-03-22|2004-06-29|Hewlett-Packard Development Company, L.P.|Automatic table detection method and system|
    US6915484B1|2000-08-09|2005-07-05|Adobe Systems Incorporated|Text reflow in a structured document|
    JP4181310B2|2001-03-07|2008-11-12|昌和 鈴木|Formula recognition apparatus and formula recognition method|
    JP2003256679A|2002-02-27|2003-09-12|Tomiko Maruta|Internet sale system|
    US20040205568A1|2002-03-01|2004-10-14|Breuel Thomas M.|Method and system for document image layout deconstruction and redisplay system|
    JP4181327B2|2002-03-06|2008-11-12|株式会社東芝|Formula recognition apparatus and formula recognition method|
    AU2002952711A0|2002-11-18|2002-11-28|Typefi Systems Pty Ltd|A method of formatting documents|
    JP4390523B2|2002-11-22|2009-12-24|オセ−テクノロジーズ・ベー・ヴエー|Segmentation of composite image by minimum area|
    TWI273443B|2003-12-09|2007-02-11|Hon Hai Prec Ind Co Ltd|System and method for converting file's format|
    US20050183033A1|2004-02-18|2005-08-18|Yaniv Feinberg|Apparatus and methods for displaying dialog box text messages including languages having different reading orders|
    US8661332B2|2004-04-30|2014-02-25|Microsoft Corporation|Method and apparatus for document processing|
    US20060001667A1|2004-07-02|2006-01-05|Brown University|Mathematical sketching|
    US7447360B2|2004-09-22|2008-11-04|Microsoft Corporation|Analyzing tabular structures in expression recognition|
    US7561737B2|2004-09-22|2009-07-14|Microsoft Corporation|Mathematical expression recognition|
    JP4607633B2|2005-03-17|2011-01-05|株式会社リコー|Character direction identification device, image forming apparatus, program, storage medium, and character direction identification method|
    DE602005002473T2|2005-07-01|2008-01-10|Pdflib Gmbh|Method for recognizing semantic units in an electronic document|
    US8249344B2|2005-07-01|2012-08-21|Microsoft Corporation|Grammatical parsing of document visual structures|
    GB2428114A|2005-07-08|2007-01-17|William Alan Hollingsworth|Data Format Conversion System|
    US20070079236A1|2005-10-04|2007-04-05|Microsoft Corporation|Multi-form design with harmonic composition for dynamically aggregated documents|
    US7853869B2|2005-12-14|2010-12-14|Microsoft Corporation|Creation of semantic objects for providing logical structure to markup language representations of documents|
    US8064696B2|2007-04-10|2011-11-22|Microsoft Corporation|Geometric parsing of mathematical expressions|
    GB0717067D0|2007-09-03|2007-10-10|Ibm|An Apparatus for preparing a display document for analysis|
    US8280892B2|2007-10-05|2012-10-02|Fujitsu Limited|Selecting tags for a document by analyzing paragraphs of the document|
    US8121412B2|2008-06-06|2012-02-21|Microsoft Corporation|Recognition of tabular structures|
    CN101329731A|2008-06-06|2008-12-24|南开大学|Automatic recognition method pf mathematical formula in image|
    US8285049B2|2008-06-06|2012-10-09|Microsoft Corporation|Corrections for recognizers|
    US8832549B2|2009-01-02|2014-09-09|Apple Inc.|Identification of regions of a document|
    US8249356B1|2009-01-21|2012-08-21|Google Inc.|Physical page layout analysis via tab-stop detection for optical character recognition|
    US8209600B1|2009-05-26|2012-06-26|Adobe Systems Incorporated|Method and apparatus for generating layout-preserved text|
    US8271873B2|2009-10-30|2012-09-18|International Business Machines Corporation|Automatically detecting layout of bidirectional text|
    US8922582B2|2009-11-16|2014-12-30|Martin J. Murrett|Text rendering and display using composite bitmap images|
    US8594422B2|2010-03-11|2013-11-26|Microsoft Corporation|Page layout determination of an image undergoing optical character recognition|
    US9218322B2|2010-07-28|2015-12-22|Hewlett-Packard Development Company, L.P.|Producing web page content|
    US8340425B2|2010-08-10|2012-12-25|Xerox Corporation|Optical character recognition with two-pass zoning|
    CN102375988B|2010-08-17|2013-12-25|富士通株式会社|File image processing method and equipment|
    JP5193263B2|2010-10-21|2013-05-08|シャープ株式会社|Document generation apparatus, document generation method, computer program, and recording medium|
    US9710435B2|2010-10-29|2017-07-18|P. Karl Halton|Object-field-based mathematics system|
    US8549399B2|2011-01-18|2013-10-01|Apple Inc.|Identifying a selection of content in a structured document|
    US20120185788A1|2011-01-19|2012-07-19|Microsoft Corporation|User interface with vertical text elements for an east-asian defined layout|
    US8910039B2|2011-09-09|2014-12-09|Accenture Global Services Limited|File format conversion by automatically converting to an intermediate form for manual editing in a multi-column graphical user interface|
    CN102411707A|2011-10-31|2012-04-11|世纪龙信息网络有限责任公司|Method and device for identifying text in picture|
    US9098471B2|2011-12-29|2015-08-04|Chegg, Inc.|Document content reconstruction|
    WO2013110287A1|2012-01-23|2013-08-01|Microsoft Corporation|Vector graphics classification engine|
    KR101812380B1|2012-01-23|2017-12-26|마이크로소프트 테크놀로지 라이센싱, 엘엘씨|Formula detection engine|
    US8559718B1|2012-04-27|2013-10-15|Abbyy Development Llc|Defining a layout of text lines of CJK and non-CJK characters|
    US9471550B2|2012-10-16|2016-10-18|Linkedin Corporation|Method and apparatus for document conversion with font metrics adjustment for format compatibility|
    US9460089B1|2012-11-07|2016-10-04|Amazon Technologies, Inc.|Flow rendering of annotation characters|
    US9330070B2|2013-03-11|2016-05-03|Microsoft Technology Licensing, Llc|Detection and reconstruction of east asian layout features in a fixed format document|
    US20140258852A1|2013-03-11|2014-09-11|Microsoft Corporation|Detection and Reconstruction of Right-to-Left Text Direction, Ligatures and Diacritics in a Fixed Format Document|KR101812380B1|2012-01-23|2017-12-26|마이크로소프트 테크놀로지 라이센싱, 엘엘씨|Formula detection engine|
    WO2013110286A1|2012-01-23|2013-08-01|Microsoft Corporation|Paragraph property detection and style reconstruction engine|
    US9946690B2|2012-07-06|2018-04-17|Microsoft Technology Licensing, Llc|Paragraph alignment detection and region-based section reconstruction|
    US20140115447A1|2012-10-22|2014-04-24|Apple Inc.|Centering Mathematical Objects in Documents|
    KR102061798B1|2012-12-20|2020-01-03|삼성전자주식회사|Method for calculating formula and an electronic device thereof|
    US9330070B2|2013-03-11|2016-05-03|Microsoft Technology Licensing, Llc|Detection and reconstruction of east asian layout features in a fixed format document|
    US9569418B2|2014-06-27|2017-02-14|International Busines Machines Corporation|Stream-enabled spreadsheet as a circuit|
    US10007943B2|2014-12-09|2018-06-26|Minted, Llc|Vendor website GUI for marketing greeting cards and envelopes|
    CN104572577B|2014-12-17|2018-09-04|百度在线网络技术（北京）有限公司|Mathematical formulae processing method and processing device|
    WO2017031716A1|2015-08-26|2017-03-02|北京云江科技有限公司|Method for analyzing and recognizing handwritten mathematical formula structure in natural scene image|
    US10540424B2|2017-06-13|2020-01-21|Microsoft Technology Licensing, Llc|Evaluating documents with embedded mathematical expressions|
    US20190139280A1|2017-11-06|2019-05-09|Microsoft Technology Licensing, Llc|Augmented reality environment for tabular data in an image feed|
    US10482162B2|2017-11-30|2019-11-19|International Business Machines Corporation|Automatic equation transformation from text|
    CN111103987B|2018-10-29|2021-06-04|北京新唐思创教育科技有限公司|Formula input method and computer storage medium|
    US11106858B2|2020-01-16|2021-08-31|Adobe Inc.|Merging selected digital point text objects while maintaining visual appearance fidelity|
    US11244203B2|2020-02-07|2022-02-08|International Business Machines Corporation|Automated generation of structured training data from unstructured documents|
    法律状态:
    2017-12-12| B25A| Requested transfer of rights approved|Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC (US) |
    2018-12-18| B06F| Objections, documents and/or translations needed after an examination request according [chapter 6.6 patent gazette]|
    2020-04-14| B06U| Preliminary requirement: requests with searches performed by other patent offices: procedure suspended [chapter 6.21 patent gazette]|
    2021-04-27| B09A| Decision: intention to grant [chapter 9.1 patent gazette]|
    2021-07-06| B16A| Patent or certificate of addition of invention granted [chapter 16.1 patent gazette]|Free format text: PRAZO DE VALIDADE: 20 (VINTE) ANOS CONTADOS A PARTIR DE 23/01/2012, OBSERVADAS AS CONDICOES LEGAIS. |
    优先权:
    申请号 | 申请日 | 专利标题
    PCT/EP2012/000285|WO2013110285A1|2012-01-23|2012-01-23|Formula detection engine|
    [返回顶部]