Browse code

xwddec: prevent overflow of lsize * avctx->height

This is used to check if the input buffer is large enough, so if this
overflows it can cause a false negative leading to a segmentation fault
in bytestream2_get_bufferu.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9d38f06d05efbb9d6196c27668eb943e934943ae)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Andreas Cadhalpun authored on 2015/12/19 03:28:51
Showing 1 changed files
... ...
@@ -141,7 +141,7 @@ static int xwd_decode_frame(AVCodecContext *avctx, void *data,
141 141
         return AVERROR_INVALIDDATA;
142 142
     }
143 143
 
144
-    if (bytestream2_get_bytes_left(&gb) < ncolors * XWD_CMAP_SIZE + avctx->height * lsize) {
144
+    if (bytestream2_get_bytes_left(&gb) < ncolors * XWD_CMAP_SIZE + (uint64_t)avctx->height * lsize) {
145 145
         av_log(avctx, AV_LOG_ERROR, "input buffer too small\n");
146 146
         return AVERROR_INVALIDDATA;
147 147
     }