Browse code

python3: Fix of CVE-2018-1060 and CVE-2018-1061

The fix for CVE-2018-1060 and CVE-2018-1061, addressing catastrophic backtracking related vulnerabilities in apop() method of pop3lib and difflib.IS_LINE_JUNK method. These vulnerabilities can result in DoS attacks.

Change-Id: I6847df920202d70008ec7a94b099985bc4a35f18
Reviewed-on: http://photon-jenkins.eng.vmware.com:8082/5504
Tested-by: gerrit-photon <photon-checkins@vmware.com>
Reviewed-by: Sharath George

dweepadvani authored on 2018/08/18 10:43:08
Showing 2 changed files
1 1
new file mode 100644
... ...
@@ -0,0 +1,158 @@
0
+From 937ac1fe069a4dc8471dff205f553d82e724015b Mon Sep 17 00:00:00 2001
1
+From: Ned Deily <nad@python.org>
2
+Date: Sun, 11 Mar 2018 14:29:05 -0400
3
+Subject: [PATCH] [3.5] bpo-32981: Fix catastrophic backtracking vulns
4
+ (GH-5955) (#6034)
5
+
6
+* Prevent low-grade poplib REDOS (CVE-2018-1060)
7
+
8
+The regex to test a mail server's timestamp is susceptible to
9
+catastrophic backtracking on long evil responses from the server.
10
+
11
+Happily, the maximum length of malicious inputs is 2K thanks
12
+to a limit introduced in the fix for CVE-2013-1752.
13
+
14
+A 2KB evil response from the mail server would result in small slowdowns
15
+(milliseconds vs. microseconds) accumulated over many apop calls.
16
+This is a potential DOS vector via accumulated slowdowns.
17
+
18
+Replace it with a similar non-vulnerable regex.
19
+
20
+The new regex is RFC compliant.
21
+The old regex was non-compliant in edge cases.
22
+
23
+* Prevent difflib REDOS (CVE-2018-1061)
24
+
25
+The default regex for IS_LINE_JUNK is susceptible to
26
+catastrophic backtracking.
27
+This is a potential DOS vector.
28
+
29
+Replace it with an equivalent non-vulnerable regex.
30
+
31
+Also introduce unit and REDOS tests for difflib.
32
+
33
+Co-authored-by: Tim Peters <tim.peters@gmail.com>
34
+Co-authored-by: Christian Heimes <christian@python.org>.
35
+(cherry picked from commit 0e6c8ee2358a2e23117501826c008842acb835ac)
36
+---
37
+ Lib/difflib.py                                |  2 +-
38
+ Lib/poplib.py                                 |  2 +-
39
+ Lib/test/test_difflib.py                      | 22 ++++++++++++++++++-
40
+ Lib/test/test_poplib.py                       | 12 +++++++++-
41
+ Misc/ACKS                                     |  1 +
42
+ .../2018-03-02-10-24-52.bpo-32981.O_qDyj.rst  |  4 ++++
43
+ 6 files changed, 39 insertions(+), 4 deletions(-)
44
+ create mode 100644 Misc/NEWS.d/next/Security/2018-03-02-10-24-52.bpo-32981.O_qDyj.rst
45
+
46
+diff --git a/Lib/difflib.py b/Lib/difflib.py
47
+index 076bbac01dee..b4ec33505644 100644
48
+--- a/Lib/difflib.py
49
+@@ -1083,7 +1083,7 @@ def _qformat(self, aline, bline, atags, btags):
50
+ 
51
+ import re
52
+ 
53
+-def IS_LINE_JUNK(line, pat=re.compile(r"\s*#?\s*$").match):
54
++def IS_LINE_JUNK(line, pat=re.compile(r"\s*(?:#\s*)?$").match):
55
+     r"""
56
+     Return 1 for ignorable line: iff `line` is blank or contains a single '#'.
57
+ 
58
+diff --git a/Lib/poplib.py b/Lib/poplib.py
59
+index 516b6f060d28..2437ea0e2717 100644
60
+--- a/Lib/poplib.py
61
+@@ -308,7 +308,7 @@ def rpop(self, user):
62
+         return self._shortcmd('RPOP %s' % user)
63
+ 
64
+ 
65
+-    timestamp = re.compile(br'\+OK.*(<[^>]+>)')
66
++    timestamp = re.compile(br'\+OK.[^<]*(<.*>)')
67
+ 
68
+     def apop(self, user, password):
69
+         """Authorisation
70
+diff --git a/Lib/test/test_difflib.py b/Lib/test/test_difflib.py
71
+index ab9debf8e252..b6c8a7dd5bed 100644
72
+--- a/Lib/test/test_difflib.py
73
+@@ -466,13 +466,33 @@ def _assert_type_error(self, msg, generator, *args):
74
+             list(generator(*args))
75
+         self.assertEqual(msg, str(ctx.exception))
76
+ 
77
++class TestJunkAPIs(unittest.TestCase):
78
++    def test_is_line_junk_true(self):
79
++        for line in ['#', '  ', ' #', '# ', ' # ', '']:
80
++            self.assertTrue(difflib.IS_LINE_JUNK(line), repr(line))
81
++
82
++    def test_is_line_junk_false(self):
83
++        for line in ['##', ' ##', '## ', 'abc ', 'abc #', 'Mr. Moose is up!']:
84
++            self.assertFalse(difflib.IS_LINE_JUNK(line), repr(line))
85
++
86
++    def test_is_line_junk_REDOS(self):
87
++        evil_input = ('\t' * 1000000) + '##'
88
++        self.assertFalse(difflib.IS_LINE_JUNK(evil_input))
89
++
90
++    def test_is_character_junk_true(self):
91
++        for char in [' ', '\t']:
92
++            self.assertTrue(difflib.IS_CHARACTER_JUNK(char), repr(char))
93
++
94
++    def test_is_character_junk_false(self):
95
++        for char in ['a', '#', '\n', '\f', '\r', '\v']:
96
++            self.assertFalse(difflib.IS_CHARACTER_JUNK(char), repr(char))
97
+ 
98
+ def test_main():
99
+     difflib.HtmlDiff._default_prefix = 0
100
+     Doctests = doctest.DocTestSuite(difflib)
101
+     run_unittest(
102
+         TestWithAscii, TestAutojunk, TestSFpatches, TestSFbugs,
103
+-        TestOutputFormat, TestBytes, Doctests)
104
++        TestOutputFormat, TestBytes, TestJunkAPIs, Doctests)
105
+ 
106
+ if __name__ == '__main__':
107
+     test_main()
108
+diff --git a/Lib/test/test_poplib.py b/Lib/test/test_poplib.py
109
+index bceeb93ad14a..799e40365214 100644
110
+--- a/Lib/test/test_poplib.py
111
+@@ -300,9 +300,19 @@ def test_noop(self):
112
+     def test_rpop(self):
113
+         self.assertOK(self.client.rpop('foo'))
114
+ 
115
+-    def test_apop(self):
116
++    def test_apop_normal(self):
117
+         self.assertOK(self.client.apop('foo', 'dummypassword'))
118
+ 
119
++    def test_apop_REDOS(self):
120
++        # Replace welcome with very long evil welcome.
121
++        # NB The upper bound on welcome length is currently 2048.
122
++        # At this length, evil input makes each apop call take
123
++        # on the order of milliseconds instead of microseconds.
124
++        evil_welcome = b'+OK' + (b'<' * 1000000)
125
++        with test_support.swap_attr(self.client, 'welcome', evil_welcome):
126
++            # The evil welcome is invalid, so apop should throw.
127
++            self.assertRaises(poplib.error_proto, self.client.apop, 'a', 'kb')
128
++
129
+     def test_top(self):
130
+         expected =  (b'+OK 116 bytes',
131
+                      [b'From: postmaster@python.org', b'Content-Type: text/plain',
132
+diff --git a/Misc/ACKS b/Misc/ACKS
133
+index 1a35aad66ce7..72c5d740bdd2 100644
134
+--- a/Misc/ACKS
135
+@@ -341,6 +341,7 @@ Kushal Das
136
+ Jonathan Dasteel
137
+ Pierre-Yves David
138
+ A. Jesse Jiryu Davis
139
++Jamie (James C.) Davis
140
+ Merlijn van Deen
141
+ John DeGood
142
+ Ned Deily
143
+diff --git a/Misc/NEWS.d/next/Security/2018-03-02-10-24-52.bpo-32981.O_qDyj.rst b/Misc/NEWS.d/next/Security/2018-03-02-10-24-52.bpo-32981.O_qDyj.rst
144
+new file mode 100644
145
+index 000000000000..9ebabb44f91e
146
+--- /dev/null
147
+@@ -0,0 +1,4 @@
148
++Regexes in difflib and poplib were vulnerable to catastrophic backtracking.
149
++These regexes formed potential DOS vectors (REDOS). They have been
150
++refactored. This resolves CVE-2018-1060 and CVE-2018-1061.
151
++Patch by Jamie Davis.
... ...
@@ -1,7 +1,7 @@
1 1
 Summary:        A high-level scripting language
2 2
 Name:           python3
3 3
 Version:        3.5.5
4
-Release:        1%{?dist}
4
+Release:        2%{?dist}
5 5
 License:        PSF
6 6
 URL:            http://www.python.org/
7 7
 Group:          System Environment/Programming
... ...
@@ -13,6 +13,7 @@ Patch0:         cgi3.patch
13 13
 Patch1:         sockWarning.patch
14 14
 Patch3:         python3-CVE-2018-1000117.patch
15 15
 Patch4:         python3-CVE-2017-18207.patch
16
+Patch5:         python3-CVE-2018-1061.patch
16 17
 BuildRequires:  pkg-config >= 0.28
17 18
 BuildRequires:  bzip2-devel
18 19
 BuildRequires:  ncurses-devel >= 6.0-3
... ...
@@ -93,6 +94,7 @@ to build python programs.
93 93
 %patch1 -p1
94 94
 %patch3 -p1
95 95
 %patch4 -p1
96
+%patch5 -p1
96 97
 
97 98
 %build
98 99
 export OPT="${CFLAGS}"
... ...
@@ -197,6 +199,8 @@ rm -rf %{buildroot}/*
197 197
 %{_bindir}/idle*
198 198
 
199 199
 %changelog
200
+*   Fri Aug 17 2018 Dweep Advani <dadvani@vmware.com> 3.5.5-2
201
+-   Fix CVE-2018-1060 and CVE-2018-1061
200 202
 *   Fri May 11 2018 Xiaolin Li <xiaolinl@vmware.com> 3.5.5-1
201 203
 -   Upgrading to 3.5.5
202 204
 *   Wed Apr 18 2018 Xiaolin Li <xiaolinl@vmware.com> 3.5.4-2