Browse code

lzo: fix overflow checking in copy_backptr()

The check `src > dst' in the form `&c->out[-back] > c->out' invokes
pointer overflow, which is undefined behavior in C.

Remove the check. Also replace `&c->out[-back] < c->out_start' with
a safe form `c->out - c->out_start < back' to avoid overflow.

CC: libav-stable@libav.org

Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

(cherry picked from commit ca6c3f2c53be70aa3c38e8f1292809db89ea1ba6)

Xi Wang authored on 2013/03/15 19:59:22
Showing 1 changed files
... ...
@@ -110,9 +110,8 @@ static inline void copy(LZOContext *c, int cnt)
110 110
  */
111 111
 static inline void copy_backptr(LZOContext *c, int back, int cnt)
112 112
 {
113
-    register const uint8_t *src = &c->out[-back];
114 113
     register uint8_t *dst       = c->out;
115
-    if (src < c->out_start || src > dst) {
114
+    if (dst - c->out_start < back) {
116 115
         c->error |= AV_LZO_INVALID_BACKPTR;
117 116
         return;
118 117
     }