This service detects hard cuts and dissolves within a video file.

Overview:

• The detection of hard cuts (or shot boundaries) is performed though the selection of frames associated to abrupt changes in the visual content of the input video sequence. The visual content is modeled by the frames' greyscale histograms, and abrupt changes are detected when the local temporal evolution of histograms exceeds an adaptive threshold depending on the cumulative histogram distance of its close temporal context. This threshold involves an offset 'c' and a scale factor 'α' on the cumulative distance which can be tuned if needed (see the Parameters section). For more information, please refer to [1], pp. 31-33.
• The detection of dissolves consists in a supervised classification of the frames using a multilayer Perceptron trained on positive and negative examples of dissolves. A dissolve is assumed to show small pixel-wise distances between consecutive frames and a large cumulative distance, as a result of a progressive transition between two shots. More information on the features used can be found in [1], pp. 35-37.

[1] Manolis Delakis, "Multimodal Tennis Video Structure Analysis with Segment Models", PhD Thesis, Université de Rennes 1, France, October 2006 (Online version).

File formats:

Vidseg takes a video file in input, and outputs a JSON file containing the frame number associated to hard cuts and dissolves.

• inputs:
  • video file: all video files supported by AVconv. If the extensions are not avi, mpg or mp4 (lowercase), a mp4 encoding will be done using AVconv.
  • JSON file (optional): uploading the JSON output "<video_file_name>.json" from A||go's multimedia webservices leads to its update with Vidseg's results under the 'vidseg' label, along with metadata from the video stream.
• output: JSON file <input_file_name>.json with the following content:

  {
  "general_info":{
  "src":"<input_file_name>",
  "video":{
  "duration":"<time_in_hh:mm:ss_format>",
  "start":"<temporal_offset_in_seconds>",
  "format":"<format>",
  "resolution":"<width>x<height> pix",
  "frame_rate":"<frame_rate> fps",
  "bit_rate":"<bit rate> kb/s",
  "color_space":"<color_space_ref>"
  }
  },
  "vidseg":{
  "annotation_type":"shot transitions",
  "system":"vidseg (<profile>)",
  "parameters":"<input_parameters>",
  "modality":"video",
  "time_unit":"frames",
  "events":[
  {
  "start": <frame_num_start>,
  "end": <frame_num_end>,
  "type": <type_of_transition>
  },
  {
  "start": <frame_num_start>,
  "end": <frame_num_end>,
  "type": <type_of_transition>
  },
  ...
  ]
  }
  }
  

  where <profile> is the cut detection thresholding profile selected. <type_of_transition> can take the labels "cut" or "grad" (dissolve) regarding the type of transition detected. <frame_num_end> and <frame_num_start> are the numbers of frames respectively associated to the beginning and the end of the transition.

Parameters:

The automatic thresholding can be adjusted using one of the two methods described below:

• -p <profile>: enables one of the following thresholding profiles for the cut detection process:
  'default' (α=1.8, c=0.10),
  'relax' (α=2.2, c=0.15),
  'aggressive' (α=2, c=0.15) and
  'custom' <value_of_α> <value_of_c> (the user choses the values of α and c).
  If unspecified, this parameter is set to 'default'.

Credits and license:

VidSeg was developed by Manolis Delakis and Sébastien Campion in IRISA/Inria Rennes. It is the property of IRISA and Inria and can be supplied under license on a case-by-case basis. It is registered at the Agency for Program Protection (APP) in France under the reference n°IDDN.FR.001.250009.000.S.P.2009.000.40000. This piece of software relies on OpenCV's python libraries, FFMPEG libraries and AVconv.