• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    Abstract: The invention relates to an imaging system parallelizing compressive sensing (CS). The system comprises a linear detector array (109,211) resolving image information along its extent with the help of focusing the incoming radiation on the detector pixels using astigmatic optics (108,212) and in that the image direction perpendicular to the extent of the detector array is resolved by the use of a number of spatial patterns on the spatial light modulator together with compressive sensing processing.
    公开号:SE1400400A1
    申请号:SE1400400
    申请日:2014-08-21
    公开日:2016-02-22
    发明作者:Markus Henriksson
    申请人:Totalförsvarets Forskningsinstitut；
    IPC主号:
    专利说明:

    4 The present invention solves the problem of long measurement times in compressed sensing by parallelizing the measurement using astigmatic optics and a linear de- tector array in the way that is evident from the following independent claim. The remaining claims concern advantageous embodiments of the invention.
    The invention will in the following be described with reference to the accompanying drawings, in which: Fig. 1 is an illustration of an embodiment of the invention where the scene is imaged onto a spatial light modulator (SLM) using standard imaging optics. The SLM imposes a line pattern mask onto the image. Each row of SLM pixels is then re-imaged onto one pixel of a linear detector array using astigmatic optics and Fig. 2 is an illustration of an embodiment of the invention where the pattern is created by the illumination source and an astigmatic camera lens images the scene onto a linear array detector.
    In many more complex imaging systems fabrication of large array detectors is a problem. It may be simply a problem of manufacturing technology where large detectors would have low yield and very high cost, as for e.g. infrared imaging. It may also be a problem of complex electronics necessary for every pixel, as in 3D laser radar detectors. In a linear detector array the electronics can expand to the sides without increasing the pixel pitch along the array dimension. This is of course not possible in a 2D detector. Another situation where 2D detector arrays are difficult is hyper-spectral imaging where the spectrum needs to be resolved in addition to the two spatial dimensions. Here it is common to use a 2D detector for the spectral and one spatial dimension and scan the second spatial dimension. CS using astigmatic optics could improve the efficiency of this setup, by removing the need to scan the slit-shaped field of view.
    Current DMD technology allows 1920x1080 pixels with 23148 Hz frame rate and 10.8 pm pixel pitch (Texas Instruments chipset 0.95 1080p). The size of DMD arrays is expected to continue to increase. If the full DMD is used for a single CS measure- ment the number of dimensions will be very high (2073600), causing the need for many measurements and hence slow frame rates. By using a linear detector array with 1x1080 pixels and astigmatic optics this is reduced to 1080 CS measurements, each with 1920 dimensions. This is a very reasonable problem size where each reconstructed frame can be collected with fifty to a few hundred DMD patterns, using integration times of 10-200 ps for each mirror pattern, and hence a frame rate of around 100 Hz can be achieved for low information content scenes and good illumi- nation conditions. For lower illumination levels longer integration times for each mirror pattern can be used to acquire the signal at the cost of lower frame rates. There is basically no limit to what integration times can be used, it only depends on the dynamic range of the detector and the light conditions. For an active illumination system multiple laser pulses can be used for the same mirror pattern and the signals added to improve the signal to noise ratio. For moderately complex scenes the compressed sensing algorithm will need a larger number of mirror patterns, but the method may be of advantage compared to classical scanning up to over 50 % of the number of dimensions.
    The smaller pixel pitch of the DMD makes long focal length imaging lenses unnecessary, potentially reducing the overall size of the imaging system even with the increased complexity of the CS setup compared to a normal camera.
    In a preferred embodiment, illustrated in Fig. 1, suitable for passive imaging, e.g. infrared imaging, but also for active 3D imaging with pulsed laser illumination, the invention is an imaging detector where the varying pattern used for the compressed sensing (CS) processing is applied in the detection system. The imaging system consists of a lens system imaging the scene onto a spatial light modulator (SLM) comprising NxP pixels. Different patterns are applied to the SLM where the pixels direct the radiation into a further re-imaging system or block the radiation depending on the pixel values in the pattern applied to the SLM. In a preferred embodiment all P rows would use the same patterns, but different patterns for different rows are also possible. The re-imaging system comprises astigmatic optical elements so that the radiation from each row of N pixels of the SLM is collected onto different pixels in a P pixel linear detector array. In this way P simultaneous measurements are performed for each pattern on the SLM and M patterns will produce data to solve P different underdetermined linear equation systems with a Mx N matrix describing each equation system. 6 In one preferred embodiment the SLM is a digital micro-mirror device (DMD). Other possibilities for the SLM include pixelated liquid crystal cells.
    The illustration in Fig. 1 shows an imaging system that studies a field of view 101.
    The scene inside the field of view could be illuminated by a light source included in the system, be illuminated by ambient light from e.g. the sun, or the thermal radiation from the objects in the scene can be used as light source. If a dedicated light source is included this could be e.g. a pulsed laser for 3D-imaging or a super-continuum laser for hyper-spectral imaging. This scene is imaged by optics 102 onto an SLM 103. The optics 102 could be a standard camera lens or a telescope suitable for the wavelength of interest. The optics images a small area 104 onto one position 105 on the SLM and other areas 106 onto other positions of the SLM 107, just like regions of the scene are imaged onto pixels of a CCD detector in a standard camera. A second astigmatic optical system 108 images the radiation reflected from or transmitted by the SLM 103 onto a linear detector array 109. The SLM is used to create patterns of vertical lines 110 on the SLM 103 where all or none of the radiation is directed towards the linear detector array 109 based on if that line on the SLM is assigned 1 or 0 in the pattern mask. The astigmatic optical system 108 images slit like regions, e.g. 111 and 113 of the SLM, that are crossing the stripe pattern 110, onto different pixels, 112 and 114 respectively, on the linear detector array 109. Different patterns 110 are used sequentially with one detector reading taken for each pattern to produce a dataset than can be used in compressed sensing reconstruction of the scene. The data from each pixel in the linear detector array produces the image of one line in the scene and these linear images are then stacked together to form a image.
    In one embodiment the astigmatic part of the re-imaging system consists of one or more cylinder lenses. In another embodiment the re-imaging system consists solely of mirrors, where a cylindrical or toroidal mirror provides the astigmatism. In one preferred embodiment for 3D imaging applications an off axis cylindrical mirror is used as the astigmatic re-imaging optics in such a way as to keep the time delay between SLM and detector equal for all pixels on the SLM.
    In one preferred embodiment the scene is illuminated by a pulsed laser and each pixel in the linear detector array comprises a temporally resolved detector circuit to provide 3D information about the scene through the time-of-flight laser radar 7 principle. In one embodiment this temporally resolved detector circuit is a photodiode and a sampling circuit comprising a number of memory registers to provide a dense temporal sampling of the received radiation intensity. The linear architecture of the detector array allows dense packing of the detectors along the line at the same time as there is ample space for electronics for the sampling. In another embodiment the detector array consists of a row of single photon avalanche diode (SPAD) detectors, each with separate electronics for collecting histograms of photon arrival times. This detector system comprises a time-correlated single-photon counting (TCSPC) laser radar system. The linear detector array for a TCSPC-system may also consist of other photon counting detectors, e.g. superconducting nanowire single photon detectors. In one embodiment the linear detector is the slit of a streak camera, allowing very high temporal resolution.
    In one preferred embodiment the astigmatic re-imaging system also includes a dispersive element to re-image the Nx P pixels of the SLM onto a Qx P pixel detector array, where each row of N pixels is redirected onto one row of Q pixels so that one wavelength component arrives at each of the Q pixels to produce a hyper-spectral imaging system. Every column of the Qx P pixel array is then a sensor of the type described in the monochromatic implementations of this invention. The hyper- spectral sensor can be implemented either by placing the dispersive element in front of the focus of the astigmatic re-imaging system, or in the focus with a second re-imaging system directing the light to the detector array. In one embodiment the dispersive element is a prism. In another embodiment the dispersive element is a grating.
    A simpler multispectral embodiment uses one or more chromatic beam splitters to direct the light to two or more discrete linear detector arrays.
    In one embodiment the two mirror positions of the DMD reflect radiation into two different but identical astigmatic optical system and linear detector array systems, that by subtraction of the measurement data produce a random sampling matrix (0) consisting of values -1 and 1 instead of 0 and 1. This is used to improve numerical stability in the reconstruction process and hence reduce the number of measurements necessary, following the results of Sale et al.; please cf. Sale, D., Rozell, C.
    J., Romberg, J. K., & Lanterman, A. D. (2012): Compressive ladar in realistic 8 environments. In 2012 IEEE Statistical Signal Processing Workshop (pp. 720-723), hereby incorporated by reference.
    In one preferred embodiment illustrated in Fig. 2 the patterns for compressed sensing processing are applied in the illumination source. A spatial light modulator projects a pattern of illuminated lines on the scene. A detector system comprising an astigmatic imaging system and a linear detector array is used so that the field of view of each detector is a stripe perpendicular to the illuminated lines on the target. In one embodiment the illumination source is a pulsed laser to provide 3D information about the scene.
    The illustration in Fig. 2 shows an imaging system where the light source 201 illuminates the whole field of view 202 in a pattern of vertical stripes 203. The light source includes a spatial light modulator to produce a changing set of vertical stripes. The spatial light modulator may be a DMD, and the full light source may be a standard computer projector. Light sources based on pulsed lasers, but otherwise similar to a projector, are suitable for longer ranges and 3D-imaging. The receiver subsystem consists of a linear detector array 211 and an astigmatic optical system 212. In the simplest implementation the astigmatic optical system is a cylindrical lens. More complex systems consisting of multiple lens elements or cylindrical or toroidal mirrors to improve the light collection capacity of the detector subsystem are possible. A single pixel 213 of the linear detector array will have a horizontal slit like field of view 214 crossing the stripes produced by the light source. A different pixel 215 will have a similar field of view 216 at a different vertical position in the total field of view 202.
    By performing a number of measurements with different patterns of vertical light stripes each detector element in the linear detector array will produce a set of collected data, which together with applied patterns of light stripes can be used to reconstruct the scene inside the horizontal slit seen by that detector element using compressive sensing reconstruction where the solution to a underdetermined linear equation system that maximizes the spasity of the scene is found. By adding these slit like scenes as lines in an image a two-dimensional image can be built.
    A number of other concrete embodiments of the invention are possible and obvious within the inventive concept to the skilled man implementing the invention. 9
    权利要求:
    Claims (7)
    [1] 1. An imaging device comprising a detector array (109,211) and a spatial light modulator (103), said imaging system resolving a two-dimensional area (101,202) using compressive sensing, characterised in that the detector is a linear detector array resolving image information along its extent with the help of focusing the incoming radiation on the detector pixels using astigmatic optics (108,212) and in that the image direction perpendicular to the extent of the detector array is resolved by the use of a number of spatial patterns on the spatial light modulator together with compressive sensing processing, thereby producing a number of compressive sensing problems equal to the number of pixels in the linear detector array, each with a dimension equal to the number of elements in the spatial light modulator patterns perpendicular to the extent of the detector array.
    [2] 2. An imaging system according to claim 1, characterised in that said spatial light modulator (103) creates a strip pattern (110,203) parallel to the direction of the linear detector array (109,211).
    [3] 3. An imaging system according to claim 1 or 2, characterised in that a system (201) illuminating the scene to be imaged includes the spatial light modulator.
    [4] 4. An imaging system according to claim 1 or 2, characterised in that it comprises standard imaging optics that produces an image of the scene (101) to be imaged on the spatial light modulator (103), and that light transmitted or reflected by the spatial light modulator is re-imaged onto the linear detector (109) array by the astigmatic optics (108).
    [5] 5. An imaging system according to claim 4, characterised in that the spatial light modulator (103) is a digital micro-mirror device and the imaging system comprises two sets of a linear detector array (109) and its astigmatic re-imaging optics (108), that light reflected in two directions from the digital micro-mirror device is collected by the respective linear detector arrays, and that the two detector readings from the detector arrays are subtracted one from the other to increase numerical stability.
    [6] 6. An imaging system according to anyone of the previous claim, characterised i n that the linear detector array (109,211) consists of hyper-spectral detectors implemented as a dispersive element and a two-dimensional detector array.
    [7] 7. An imaging system according to anyone of the previous claim, characterised i n that it comprises a pulsed light source (201) illuminating the scene to be imaged and that the linear detector array (211) consists of temporally resolved detectors to produce a 3D-image of the scene. Patentansokan nr / Patent application No: 1400400-6 1 fOljande bilaga finns en oversattning av patentkraven till svenska. Observera att det är patentkravens lydelse pa engelska som galler. A Swedish translation of the patent claims is enclosed. Please note that only the English claims have legal effect. I
    类似技术:
    公开号 | 公开日 | 专利标题
    SE1400400A1|2016-02-22|An imaging system parallelizing compressive sensing imaging
    Sun et al.2016|Single-pixel three-dimensional imaging with time-based depth resolution
    US10345684B2|2019-07-09|Pattern projection and imaging using lens arrays
    US9514378B2|2016-12-06|Space-time modulated active 3D imager
    WO2016015516A1|2016-02-04|Optical imaging method using single pixel detector
    RU2653772C1|2018-05-14|System for forming broadband hyperspectral image based on compressible probing with a random diffraction grating
    CN1702452B|2011-05-04|Digital microscope multi-objective imaging spectrometer apparatus
    JP2016082574A|2016-05-16|Multiple pixel pitch super resolution technology
    JP2021506168A|2021-02-18|Light field image processing method for depth acquisition
    Edgar et al.2016|Real-time 3D video utilizing a compressed sensing time-of-flight single-pixel camera
    Howland et al.2011|Compressive sensing LIDAR for 3D imaging
    US10783652B2|2020-09-22|Plenoptic imaging apparatus, method, and applications
    US10992924B2|2021-04-27|Stereo-polarimetric compressed ultrafast photography | systems and methods
    Sher et al.2018|Low intensity LiDAR using compressed sensing and a photon number resolving detector
    Velten et al.2016|Imaging the propagation of light through scenes at picosecond resolution
    RU2015119299A|2016-12-10|Optoelectronic system for determining the spectroenergetic parameters and coordinates of the infrared laser source
    Du Bosq et al.2018|An overview of joint activities on computational imaging and compressive sensing systems by NATO SET-232
    Johnson et al.2020|Single-pixel LIDAR with deep learning optimised sampling
    CN103558160B|2015-10-28|A kind of method and system improving light spectrum image-forming spatial resolution
    Edgar et al.2016|First-photon 3d imaging with a single-pixel camera
    EP3826283A1|2021-05-26|Imaging apparatus comprising at least a light source and a plenoptic camera
    Wang2018|High resolution 2D imaging and 3D scanning with line sensors
    Antoshkin et al.2012|Using photodetectors in Shack-Hartmann wavefront sensors
    Sun et al.2015|Infrared single-pixel imaging utilising microscanning
    Edgar et al.2013|3D computational ghost imaging
    同族专利:
    公开号 | 公开日
    EP3183873A1|2017-06-28|
    US20170276545A1|2017-09-28|
    EP3183873A4|2018-01-17|
    WO2016028200A1|2016-02-25|
    SE538072C2|2016-02-23|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    WO2006116134A2|2005-04-21|2006-11-02|William Marsh Rice University|Method and apparatus for compressive imaging device|
    US7336353B2|2005-10-17|2008-02-26|Duke University|Coding and modulation for hyperspectral imaging|
    US8305575B1|2008-06-23|2012-11-06|Spectral Sciences, Inc.|Adaptive spectral sensor and methods using same|
    US20110260036A1|2010-02-22|2011-10-27|Baraniuk Richard G|Temporally- And Spatially-Resolved Single Photon Counting Using Compressive Sensing For Debug Of Integrated Circuits, Lidar And Other Applications|
    US8860835B2|2010-08-11|2014-10-14|Inview Technology Corporation|Decreasing image acquisition time for compressive imaging devices|CN107749756B|2017-10-13|2020-12-18|成都正扬博创电子技术有限公司|Image signal acquisition method based on compressed sensing|
    US10158038B1|2018-05-17|2018-12-18|Hi Llc|Fast-gated photodetector architectures comprising dual voltage sources with a switch configuration|
    US10340408B1|2018-05-17|2019-07-02|Hi Llc|Non-invasive wearable brain interface systems including a headgear and a plurality of self-contained photodetector units configured to removably attach to the headgear|
    US10515993B2|2018-05-17|2019-12-24|Hi Llc|Stacked photodetector assemblies|
    US10420498B1|2018-06-20|2019-09-24|Hi Llc|Spatial and temporal-based diffusive correlation spectroscopy systems and methods|
    US11213206B2|2018-07-17|2022-01-04|Hi Llc|Non-invasive measurement systems with single-photon counting camera|
    WO2020131148A1|2018-12-21|2020-06-25|Hi Llc|Biofeedback for awareness and modulation of mental state using a non-invasive brain interface system and method|
    EP3742135B1|2019-05-20|2022-01-19|Centre National de la Recherche Scientifique|Hyperspectral time-resolved mono-pixel imaging|
    US11081611B2|2019-05-21|2021-08-03|Hi Llc|Photodetector architectures for efficient fast-gating comprising a control system controlling a current drawn by an array of photodetectors with a single photon avalanche diode|
    US10868207B1|2019-06-06|2020-12-15|Hi Llc|Photodetector systems with low-power time-to-digital converter architectures to determine an arrival time of photon at a photodetector based on event detection time window|
    US11096620B1|2020-02-21|2021-08-24|Hi Llc|Wearable module assemblies for an optical measurement system|
    US11245404B2|2020-03-20|2022-02-08|Hi Llc|Phase lock loop circuit based signal generation in an optical measurement system|
    US11187575B2|2020-03-20|2021-11-30|Hi Llc|High density optical measurement systems with minimal number of light sources|
    法律状态:
    2019-04-02| NUG| Patent has lapsed|
    优先权:
    申请号 | 申请日 | 专利标题
    SE1400400A|SE538072C2|2014-08-21|2014-08-21|An imaging system parallelizing compressive sensing imaging|SE1400400A| SE538072C2|2014-08-21|2014-08-21|An imaging system parallelizing compressive sensing imaging|
    PCT/SE2015/000048| WO2016028200A1|2014-08-21|2015-07-24|An imaging system parallelizing compressive sensing imaging|
    EP15833160.3A| EP3183873A4|2014-08-21|2015-07-24|An imaging system parallelizing compressive sensing imaging|
    US15/504,939| US20170276545A1|2014-08-21|2015-07-24|An imaging system parallelizing compressive sensing imaging|
    [返回顶部]