NC State
BioResources
Müller, T., Lenske, A., Barbier, C., Hauptmann, M., and Majschak, J.-P. (2019). "Geometry-invariant wrinkle detection in sealing rims of paperboard containers," BioRes. 14(2), 2536-2549.

Abstract

Noticeable improvements were achieved in the method for quality evaluation of formed paperboard containers. The method now allows for in situ evaluation of unavoidable wrinkle structures along the sealing rim of formed containers. An image of the sealing rim was provided. In this image, the contour of the sample was detected. The contour line was then offset to the inside of the sample, so that the new line was on the sealing rim, regardless of the original contour geometry. Along this offset contour line, the wrinkle structure was evaluated by using a previously described cross-correlation-based method. The repeatability and accuracy of the method were validated by comparing the detection results with the results from thorough human examiners. Furthermore, an approach to find the optimum settings for the wrinkle detection program is described and an outlook on implications for industrial adaptation of this method is given.


Download PDF

Full Article

Geometry-invariant Wrinkle Detection in Sealing Rims of Paperboard Containers

Tobias Müller,a,* Alexander Lenske,a Christophe Barbier,b Marek Hauptmann,a and Jens-Peter Majschak a

Noticeable improvements were achieved in the method for quality evaluation of formed paperboard containers. The method now allows for in situ evaluation of unavoidable wrinkle structures along the sealing rim of formed containers. An image of the sealing rim was provided. In this image, the contour of the sample was detected. The contour line was then offset to the inside of the sample, so that the new line was on the sealing rim, regardless of the original contour geometry. Along this offset contour line, the wrinkle structure was evaluated by using a previously described cross-correlation-based method. The repeatability and accuracy of the method were validated by comparing the detection results with the results from thorough human examiners. Furthermore, an approach to find the optimum settings for the wrinkle detection program is described and an outlook on implications for industrial adaptation of this method is given.

Keywords: Paperboard; Deep drawing; Forming; Quality evaluation; Image processing; Packaging

Contact information: a: Institute of Natural Materials Technology, Chair of Processing Machines and Technology, Technische Universität Dresden, Dresden, Germany; b: BillerudKorsnäs AB, Gruvön Mill, Sweden; *Corresponding author: tobias.mueller1@tu-dresden.de

INTRODUCTION

Through the recent works of Hauptmann and Majschak (2011), Hauptmann et al. (2014), Hauptmann et al. (2016), Hauptmann (2017), and Vishtal (2015), the geometric boundaries of press forming and deep-drawing of paperboard have been advanced from simple round cups and oval trays to complex packaging containers with concave sections and multiple geometric features. Some examples from these shapes are shown in Fig. 1.

Fig. 1. (a) Concave sections (Hauptmann et al. (2014)), (b) sharp edge radius and different radii (Hauptmann et al. (2016)), (c) concave/convex transition (Hauptmann and Majschak (2011)), (d) complex geometry (Hauptmann (2017)), and (e) with sealing rim

For an industrial application of the technology, robust process and quality control is important. A forming, filling, and sealing machine for paperboard packaging needs to reach an output of ca. 60 parts per minute to be economically sensible. Machine downtimes and quality rejects are to be minimized. A robust and inline quality evaluation is the prerequisite for such an application.

For quality evaluation and assessment of the forming process, the wrinkles in the sealing rim of the paperboard container are examined. Broad wrinkles are optically displeasing and can prevent leakproof sealing of the container (Leminen et al. 2015). This also reduces the quality output of a forming machine.

Previously, the detection and evaluation of wrinkle structures in the wall section or on the sealing rim have depended on the specific geometry and measuring setup. Müller et al. (2017) used laser topography that relies on scanning the container along a predetermined contour. The setup of Wallmeier et al. (2015) prohibited wrinkle detection on non-round geometries, and Leminen et al. (2016) required predefined regions of interest (ROIs) on the sealing rim for evaluation to work. The method described by Müller et al. (2018) was able to evaluate round and elliptical shapes, but it was limited to the observation of the wall section in two distinct ROIs.

The current study is a continuation of the previous work and attempted to comprehensively advance the basic method detailed by Müller et al. (2018). Industry requirements for a fast inline evaluation of different geometries were addressed. The goal of evaluation one sample per second was reached and meets the industry requirement that every sample can be scanned at a machine output of 60 pieces per minute. The advanced method and setup allowed for an expeditious and accurate quality evaluation of the sealing rim of formed paperboard containers. The entire sealing rim was evaluated in one step. The method was not limited to a single geometry, but was applied to a broad range of sealing rim geometries.

EXPERIMENTAL

Method and Setup for Image Generation

The evaluation chamber (Fig. 2) was comprised of a semitransparent, opaque pane, on which the sample was placed. The pane acted as a diffusor and was backlit by an LED-matrix. Above the opaque pane, a camera was placed, so that the entire size of the sealing rim of the sample could be photographed in one image.

Experiments with alternative lighting methods showed that only backlighting was robust against distortions (i.e., waviness and warping) in the sealing rim and could show the wrinkle structures with enough contrast to be discerned from the material structure itself. Tanninen et al. (2017) described a setup with backlit press forming samples as well.

Figure 3a shows an image of a sample obtained with darkfield lighting. In darkfield lighting, the light-source is positioned at a shallow angle towards the illumination plane. Generally, darkfield lighting is suitable to produce a high contrast on surface structures. It can be seen that the curvature and waviness of the sealing rim led to a strong shadowing effect on the sealing rim that prohibited sensible wrinkle detection. The image in Fig. 3b was acquired under a dome light, which granted a diffused and even light source from a half-sphere dome above the sample. Still, the sample was not lit homogenously along the edge of the sample and the contrast was not sufficient. Figure 3c shows the resulting image of a backlit sample. The wrinkle structures in the sealing rim can be seen with a high contrast, which was beneficial for the following wrinkle detection.

Fig. 2. Setup for image acquisition

Fig. 3. (a) Darkfield lighting, (b) dome-light, and (c) backlighting

The algorithm for wrinkle detection in the sealing rim was programmed in Python (Version 2.7.10.0, P, Beaverton, OR, USA), utilizing the OpenCV computer vision library (Version 2.4.11, opencv.org). The images for the wrinkle evaluation were loaded from a user-specified folder. The folder can contain an arbitrary number of images from many samples. Wrinkle evaluation was then performed in two steps: image preprocessing and wrinkle detection.

Image preprocessing

The original image was converted to an 8-bit grayscale image. This ensured faster processing times, as no information was needed for wrinkle detection because it was encoded in the RGB-channels. The edges of the image were cropped so that the sample was the only object in the image with a clear contour (Fig. 4a).

The image was then opened to remove noise and disturbances. The erosion operation was followed by a dilation operation as described by Haralick et al. (1987). The image was scanned for contours with the OpenCV function findContours, which utilized a contour detection method described by Suzuki and Abe (1985). The findContours function used a gray value-threshold to detect contours and listed all of the contours found in the image, ordered by size in pixels. The threshold was initially set to a gray value of 80, which was adjusted incrementally for stable contour detection. Of the contours, only the contour with the greatest length was kept and the rest were discarded. Contour detection could be assisted with a heuristic function regarding the expected contour length range. The length of the detected contour was measured and compared using the heuristic function. If the contour was within the range, it was used for further wrinkle detection. If the contour was outside the admissible range, the threshold value was increased and contour detection was started anew until an admissible contour was noted (Fig. 4b). This loop ensured that only the contour of the sample was detected and that contour detection was robust against the illumination variation between individual images.

Fig. 4. (a) Original image, (b) contour detection failed, and (c) preprocessed image with contour (red line)

Every pixel that was not enclosed by the contour was set to black (gray value = 0). Those pixels were outside of the sample and not of interest for detection of wrinkle structures. This image mask was then merged with the original, cropped image, which resulted in the final preprocessed image (Fig. 4c).

Wrinkle detection

The Cartesian coordinates (X, Y) of the contour line were transformed to polar coordinates (φ, ρ) and sorted by ascending angle φ. The coordinate transformation ensured that a continuous contour vector with unique φ-values was generated. To decrease the computing time, only every tenth (φ, ρ) coordinate pair was kept. The radial component ρ was decreased by a factor of 0.95 so that the contour line was on the sealing rim instead of on the outer edge of the sample. Finally, the coordinates were transformed back to Cartesian coordinates (X, Y) so that the program could identify each coordinate pair with a pixel (px, py) in the image. However, through the transformation and ordering of the coordinate pairs, now the (X, Y) coordinates were ordered so that an incremental sweep of the coordinate pairs in the contour vector was mapped to a continual sweep of the actual contour in the image.

As proposed by Müller et al. (2018), a template of an ideal wrinkle with changing grayscale values (GV(x)) was built symmetrically to the middle of the image on the X-axis (x0), according to Eq. 1, with the sharpness (sn) scaled to the size of the template itself. Figure 5 shows a visualization of the resulting template.

 (1)

Fig. 5. Template of an ideal wrinkle (Müller et al. 2018)

The template was moved along the offset contour line. For each step, the cross-correlation of the template and the section of the image on which the template was currently overlaid was calculated and saved for later, along with the current coordinates and the rotation of the template with regards to the center point of the contour line. The template was then moved to the next step, rotated by the current φ, and the process was repeated as is indicated in Fig. 6. To save computing time, the size of the iteration step could be adjusted, but was at least half the size of the template so that the same wrinkle structure was not detected multiple times.

Fig. 6. Sweeping of the template along the offset contour line

At each point, the template cross-correlation and overlapping image section were calculated and written in a new column alongside the respective coordinate vector. After the cross-correlations for the entire contour were recorded, the array with the correlation values was compared with a correlation threshold. Any values that were smaller than the threshold were then set to zero and no wrinkles were marked as detected at this coordinate pair. The Euclidian distance between the coordinates of neighboring non-zero values was calculated and compared with a minimum distance. The minimum distance ensured that the same structure was not detected multiple times. Should there have been more than one non-zero correlation value within the minimum distance, then only the largest value was kept and all other values within the minimum distance were set to zero. This guaranteed that only the best fit was kept.

The remaining non-zero values in the correlation array were counted. This yielded the amount of detected wrinkle structures. Furthermore, the standard deviation of the Euclidian distances between neighboring points that had a non-zero correlation value was calculated. As described by Hauptmann (2013), the standard deviation of the wrinkle distances can be used as a characteristic for the uniformity of the wrinkle distribution.

The wrinkle structures were marked in the original image, which was saved separately (Fig. 7). The wrinkle data for this image was written to a database where it could be compared with the material and forming parameters of the original sample.