iPhone: bloqueo de salida de captura de AVCaptureSession (AVCaptureVideoDataOutput)

Question

Estoy capturando video y convirtiéndolo en un CGImage para procesarlo. Funcionará durante ~ 10 segundos, recibirá una advertencia de memoria y luego se bloqueará (generalmente dice que los formateadores de datos no estaban disponibles temporalmente). ¿Alguien puede ayudarme a resolver el problema?

- (void) captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection { 

// CONVERT CMSAMPLEBUFFER INTO A CGIMAGE 
CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); 

CVPixelBufferLockBaseAddress(imageBuffer,0); 


uint8_t *baseAddress = (uint8_t *)CVPixelBufferGetBaseAddress(imageBuffer); 
size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer); 
size_t width = CVPixelBufferGetWidth(imageBuffer); 
size_t height = CVPixelBufferGetHeight(imageBuffer); 


CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB(); 
CGContextRef newContext = CGBitmapContextCreate(baseAddress, width, height, 8, bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst); 
CGImageRef cgimage = CGBitmapContextCreateImage(newContext); 
UIImage *sourceImage= [UIImage imageWithCGImage:cgimage scale:1.0f orientation:UIImageOrientationLeftMirrored]; 
CGImageRelease(cgimage); 
CGContextRelease(newContext); 
CGColorSpaceRelease(colorSpace); 

CVPixelBufferUnlockBaseAddress(imageBuffer,0); 


// *** 
// Adding code after this point to do image transformation usually causes it to crash 
UIImage *rot = [self scaleAndRotateImage:sourceImage]; 
self.detectImage = rot; 



} 

Código para transformar la imagen ....

- (UIImage*)scaleAndRotateImage:(UIImage *)image{ 
    int kMaxResolution = 320; // Or whatever 

    CGImageRef imgRef = image.CGImage; 

    CGFloat width = CGImageGetWidth(imgRef); 
    CGFloat height = CGImageGetHeight(imgRef); 

    CGAffineTransform transform = CGAffineTransformIdentity; 
    CGRect bounds = CGRectMake(0, 0, width, height); 
    if (width > kMaxResolution || height > kMaxResolution) { 
     CGFloat ratio = width/height; 
     if (ratio > 1) { 
      bounds.size.width = kMaxResolution; 
      bounds.size.height = bounds.size.width/ratio; 
     } 
     else { 
      bounds.size.height = kMaxResolution; 
      bounds.size.width = bounds.size.height * ratio; 
     } 
    } 

    CGFloat scaleRatio = bounds.size.width/width; 
    CGSize imageSize = CGSizeMake(CGImageGetWidth(imgRef), CGImageGetHeight(imgRef)); 
    CGFloat boundHeight; 
    UIImageOrientation orient = image.imageOrientation; 
    switch(orient) { 

     case UIImageOrientationUp: //EXIF = 1 
      transform = CGAffineTransformIdentity; 
      break; 

     case UIImageOrientationUpMirrored: //EXIF = 2 
      transform = CGAffineTransformMakeTranslation(imageSize.width, 0.0); 
      transform = CGAffineTransformScale(transform, -1.0, 1.0); 
      break; 

     case UIImageOrientationDown: //EXIF = 3 
      transform = CGAffineTransformMakeTranslation(imageSize.width, imageSize.height); 
      transform = CGAffineTransformRotate(transform, M_PI); 
      break; 

     case UIImageOrientationDownMirrored: //EXIF = 4 
      transform = CGAffineTransformMakeTranslation(0.0, imageSize.height); 
      transform = CGAffineTransformScale(transform, 1.0, -1.0); 
      break; 

     case UIImageOrientationLeftMirrored: //EXIF = 5 
      boundHeight = bounds.size.height; 
      bounds.size.height = bounds.size.width; 
      bounds.size.width = boundHeight; 
      transform = CGAffineTransformMakeTranslation(imageSize.height, imageSize.height); 
      transform = CGAffineTransformScale(transform, -1.0, 1.0); 
      transform = CGAffineTransformRotate(transform, 3.0 * M_PI/2.0); 
      break; 

     case UIImageOrientationLeft: //EXIF = 6 
      boundHeight = bounds.size.height; 
      bounds.size.height = bounds.size.width; 
      bounds.size.width = boundHeight; 
      transform = CGAffineTransformMakeTranslation(0.0, imageSize.width); 
      transform = CGAffineTransformRotate(transform, 3.0 * M_PI/2.0); 
      break; 

     case UIImageOrientationRightMirrored: //EXIF = 7 
      boundHeight = bounds.size.height; 
      bounds.size.height = bounds.size.width; 
      bounds.size.width = boundHeight; 
      transform = CGAffineTransformMakeScale(-1.0, 1.0); 
      transform = CGAffineTransformRotate(transform, M_PI/2.0); 
      break; 

     case UIImageOrientationRight: //EXIF = 8 
      boundHeight = bounds.size.height; 
      bounds.size.height = bounds.size.width; 
      bounds.size.width = boundHeight; 
      transform = CGAffineTransformMakeTranslation(imageSize.height, 0.0); 
      transform = CGAffineTransformRotate(transform, M_PI/2.0); 
      break; 

     default: 
      [NSException raise:NSInternalInconsistencyException format:@"Invalid image orientation"]; 

    } 

    UIGraphicsBeginImageContext(bounds.size); 

    CGContextRef context = UIGraphicsGetCurrentContext(); 

    if (orient == UIImageOrientationRight || orient == UIImageOrientationLeft) { 
     CGContextScaleCTM(context, -scaleRatio, scaleRatio); 
     CGContextTranslateCTM(context, -height, 0); 
    } 
    else { 
     CGContextScaleCTM(context, scaleRatio, -scaleRatio); 
     CGContextTranslateCTM(context, 0, -height); 
    } 

    CGContextConcatCTM(context, transform); 

    CGContextDrawImage(UIGraphicsGetCurrentContext(), CGRectMake(0, 0, width, height), imgRef); 
    UIImage *imageCopy = UIGraphicsGetImageFromCurrentImageContext(); 
    UIGraphicsEndImageContext(); 

    //[self setRotatedImage:imageCopy]; 
    return imageCopy; 
} 

Esta función es sólo de fondo para ver cómo configurar la salida de vídeo ...

AVCaptureVideoDataOutput *videoOut = [[AVCaptureVideoDataOutput alloc] init]; 
[videoOut setAlwaysDiscardsLateVideoFrames:YES]; 
[videoOut setVideoSettings:[NSDictionary dictionaryWithObject:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA] forKey:(id)kCVPixelBufferPixelFormatTypeKey]]; // BGRA is necessary for manual preview 
dispatch_queue_t my_queue = dispatch_queue_create("com.example.subsystem.taskXYZ", NULL); 
[videoOut setSampleBufferDelegate:self queue:my_queue]; 
if ([self.captureSession canAddOutput:videoOut]) [self.captureSession addOutput:videoOut]; 
else NSLog(@"Couldn't add video output"); 
[videoOut release]; 

Answer 1

En la creación de la salida de video, la cola de despacho recién creada no se lanza. Se puede desbloquear con

dispatch_release(queue); 

Pero yo no creo que esta función se llama demasiado a menudo, por lo que la fuga se origina probablemente en otros lugares. Mirado su código varias veces, no pude encontrar ningún otro culpable ...

¿Ha intentado buscar la fuga con la herramienta del instrumento Leaks?

Answer 2

Tuve un problema similar. Lo que terminó sucediendo fue que la cola se estaba llenando de fotogramas no procesados, porque no estaba procesando lo suficientemente rápido en el objeto delegado.

Mi solución era hacer (una vez por cuadro procesado):

proctr++; 
if ((proctr % 20) == 0) { 
    deferImageProcessing = true; 
    dispatch_sync(queue, ^{ 
    [self queueFlushed]; 
    }); 
} 

- (void)queueFlushed { 
    deferImageProcessing = false; 
} 

Luego, en el código de procesamiento de imagen real

- (void)captureOutput:(AVCaptureOutput *)captureOutput 
    didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer 
    fromConnection:(AVCaptureConnection *)connection 
{ 
    if (deferImageProcessing) 
    return; 
    // do whatever else I'm doing... 
} 

Esencialmente, ocasionalmente suspender el procesamiento de imágenes hasta que la cola se vacía.
Espero que esto sea útil.