CD measurement data from SEM images are widely used in OPC model calibration and verification ^[1-2]. As the complexity of the lithography process continues to increase, OPC model calibration using CD measurement data only is facing difficulty in obtaining model that can meet the stringent accuracy requirements. As a remedy, OPC model calibration including SEM image contours have been proposed in recent years ^[3-7]. From the information theoretic point of view, SEM image contours have significant advantages in terms of pattern coverage and sampling numbers, as a result, OPC models with better accuracy and better generalization capability are expected. Besides the gradual adoption of SEM image contours in OPC model calibration, more detailed quantitative analysis based on resist top contours and resist bottom contours can greatly facilitate lithography hotspot detection with improved certainty.

In advanced low K₁ lithography processes, SEM images are intrinsically low contrast. The situation exacerbates when only few scanning frames can be averaged in order to avoid resist shrinkage from e-beam exposure. This presents enormous challenges for contour extraction from such SEM images. The classical methods utilizing edge detection operators ^[8], such as Sobel, Prewit, log, etc., are incapable of extracting contours from low contrast SEM images with robustness. The Canny algorithm ^[9], on the other hand, is capable of extracting contours with better robustness, however, it lacks the ability to distinguish resist top contours from resist bottom contours. The limitation of the classical image contour extraction methods lies in its use of information. All classical methods use very local information only for contour extraction. To overcome the limitation, new algorithms should be developed, in which information beyond local can be made use of fully. In this respect, DCNN is considered to be the best choice ^[10-11], because it has proven itself to possess capabilities of exploring spatial information hierarchically on a large spatial scale. Furthermore, SEM images from lithography process only have three distinct areas, namely, the resist top areas, the substrate areas, and the resist edge areas of varying slopes. This simplicity suggests the possibility of employing DCNN to develop contour extraction models that are capable of separating resist top contours from resist bottom contours. Besides its capability in exploring spatial information, DCNN, as a machine learning model, is fully compatible with parallel computation architecture, making full chip SEM image contour extraction feasible. To develop such DCNN based SEM image contour extraction models, we have prepared a set of SEM images together with their resist top contours and resist bottom contours, and use a special DCNN structure to train the contour extraction models. The special DCNN structure possesses characteristics as of being wide in receptive field, moderate in network depth, and high in resolution.

Several contour extraction methods for post-lithography SEM images have been developed in commercial available tools, such as DesignGauge-Analyzer (DG-A, Hitachi) ^{[6, 7]}, ASELTA solution ^[12], Canny-improved method ^[5], etc. Put the contour extraction robustness aside, these methods do not offer capability of separating resist top contours from resist bottom contours. To offer such a capability and full chip operation capability, we have proposed a method based on deep learning architecture for SEM contour extraction, which is inspired by the enormous success achieved so far from deep learning technology in various computer vision areas, such as classification, object detection and semantic segmentation ^[13-15]. DCNN is superior to the traditional methods because it has large receptive field. Empirically, a large receptive field is able to achieve a better sparse representation for image based on the image’s priors of nonlocal self-similarity (NSS) ^[16]. From the information theoretic point of view, the sparser the image representation is, the less the information loss is for image restoration ^[17].

The neural network architecture is shown in Figure 1. The first layer is reversible down-sampling operation, used to lower the dimensions for input SEM image. The split sub-images are arranged in a stack format as the input of the subsequent module (plain Deep Convolution Neural Network, DCNN). For instance, if the down-sampling factor is 2, then the input image of DCNN is: M/2×N/2×4C, where M and N are the height and width of the original SEM image, and C represents the number of channels in original image. Since SEM images are gray scale images, therefore, C is equal to 1 in our study.

The DCNN module acts as a non-linear mapping from gray scale SEM image to contour image. In order to learn the mapping function, the DCNN module learns features from input image with convolutional operators. The first layer has convolutional operation followed by rectified linear unit (ReLU) activation function operation; the last layer has convolutional operation only; while other layers have convolutional operation followed by batch normalization (BN) ^[18] and ReLU activation function operation. Zero padding is used to maintain feature map image size for each intermediate layer. The final size of the output from DCNN module is M/2×N/2×4C. The final image is constructed from the images of the 4C channels through up-sampling operation to restore the original image size M×N×C. The total number of layers is 15, the number of channels in each layer is 64, the convolutional kernel size is 3x3, and the image area used for cost function calculation is the center area of each image with size 128x128. The non-central image area is excluded from cost function calculation, because those areas are corrupted by zero padding and convolutional operation.

The down-sampling operation does the trick for the subsequent DCNN to explore wider spatial information with shallower network without the use of residual connections ^[19]. Residual connections are designed to facilitate the training of very deep convolutional neural network ^[20], however, models of good performance can still be achieved using plain neural network of moderate depth by advanced optimization techniques such as BN and Adam optimizer ^[21].

To train the DCNN network for contour extraction, we need to prepare a high quality training dataset. The original SEM images of 28nm technology node are adopted and the pixel size is 1.32nm, the SEM images are often noisy, and some SEM images even have patterning failures of varying degree of severity, such as scums, neckings, bridgings. To obtain the contours from such a SEM image, the SEM image must first go through a denoising process, followed by contour extraction operation using in-house refinement tool.

The denoising operation should remove the noise as much as possible while at the same time retaining signal information (details) as much as possible. A sophisticated DCNN denoising model has been applied to achieve the satisfactory results. The residual noise map shows no noticeable structure in it, suggesting that the detailed structural information of the original image is retained successfully in the denoised image, as shown in Figure 2.

In-house contour refinement tool is developed to further process the raw contours from Canny algorithm. The contour refinement operations include connecting the broken contours when the broken area is small, removing redundant fake contour segments. The refined contours are classified into two classes: one is the top resist contour class and the other is the resist bottom contour class. Each class dataset is split into three sub datasets for training, verification and test with the ratio 6:2:2. The total image number for training is around 6k. The batch size used for training is 16.

The bottom and top contour models are trained independently based on the prepared datasets. The learning rate is set at 5e-4, and the training error patient is fixed at 20 (if the training error stays unchanged in 20 consecutive epochs, then the parameters of each batch are merged to form the model). Other parameters are used as default in Adam. Data augmentation such as flip and rotation are adopted to expand the dataset. The bottom and top contour extraction model training take ~10 and 18 hours with single GPU, respectively. Although the model training takes considerable amount of time, the test is very fast with a speed of ~ 50ms/frame using single CPU only and ~20ms/frame using single GPU.

The test dataset includes post lithography SEM images with necking and bridging of varying degrees, some examples are shown in Figure 3. Because there is no universal standard for contour definition from SEM images, robustness and consistency are more important for a contour extraction algorithm, since dimension difference between contours extracted from different algorithms can be calibrated against each other easily as long as both algorithms possess internal consistency. From a practical point of view, contour extraction algorithm assessment should focus on the following aspects: i) it does not miss contour in a visually distinct boundary; ii) it does not produce fake contour in the background or in the foreground; iii) it has the flexibility and capability of repairing broken contours if the broken segments gap is small; iv) it should be fast enough to support Fab production needs.

The training and test results of the top resist contour model and the resist bottom contour model are summarized in Table 1. Areas with relatively large “broken” gap between contour segments are identified as “broken” contours; and contours that are not at feature edges are identified as “false” contours. The results indicate that the bottom contour extraction model can achieve much better result than the top contour extraction model in training, validation and test, with only few percentage of “broken” and “false” contour areas. By no means does this observation implies that the bottom contour extraction model is trained better or perform better, rather, this is because the boundary between resist and substrate is usually defined better than the boundary between the resist sidewall and the resist top surface (often rounded). The roundness of the resist top boundary leads to more ambiguity for the contour extraction model to judge whether an image pixel belongs to contour class or not. The high percentage of “broken” contours or “false” contours existing in resist top contour set is entirely due to blurred boundaries inherited in the original SEM images.

4.1. Top & Bottom Contour Extraction Model Performance

A typical result for contour extraction is illustrated in Figure 4. A general two dimensional (2D) pattern with considerable noise and some distortions is shown in Figure 4 a). Figure 4 c) and 4 d) are the top (white) and the bottom (yellow) contour results, respectively. Figure 4 b) shows the overlaid top and bottom contours on the original image, from which high fidelity and consistency of the contours relative to the original image structures are clearly demonstrated.

DCNN-based method can outperform traditional image contour extraction methods in two aspects with significantly large margin: suppressing the redundant contours and “repairing” broken contours within a local area, as shown in Figure 5. Although Canny algorithm is the landmark in image contour extraction because of its simplicity and efficiency, it often fails to achieve satisfactory results when an image is not in good condition, e.g. the boundary is not clear or continuous locally. Detailed comparison of contour extraction performance between DCNN-based method and the Canny method is shown in Figures 5 a) – 5 d). It should be understood that ‘broken’ and ‘false contours could exist in extracted contour sets despite DCNN-based method’s great versatility, as shown in Figure 6. Those “unsuccessful” contour extraction areas are very often indicative of lithography patterning failures, which can be easily identified and marked for further visual inspection.

A specially designed DCNN structure is proposed for lithography SEM image contour extraction. The DCNN SEM image contour extraction models outperform the prevalent traditional Canny method in a significant large margin in terms of robustness. When trained separately, the DCNN contour extraction models are capable of separating resist top contours from resist bottom contours. This capability has paved the way of estimating resist sidewall angles at any location, it can also greatly facilitate the identification of “soft defects” (the defect with high inconsistency of judgments from different lithography engineers) ^[24] in a lithography SEM image where contours are broken. Together with sophisticated image alignment algorithm, the statistics of the process variation band, the critical dimension, the edge placement variation, etc., can all be estimated on a full chip scale.