odd code in requests() regarding json decoding #302

New issue

Closed

opened 2014-01-11 12:34:18 -08:00 by cash · 0 comments

cash commented

2014-01-11 12:34:18 -08:00

(Migrated from github.com)

Here is the code (with unrelated stuff left out):

        content = response.content.decode('utf-8')

        #  Wrap the json loads in a try, and defer an error
        #  Twitter will return invalid json with an error code in the headers
        json_error = False
        try:
            try:
                # try to get json
                content = content.json()
            except AttributeError:
                # if unicode detected
                content = json.loads(content)
        except ValueError:
            json_error = True
            content = {}

response is a requests.Response object. response.content is of type str in Python 2.7 and of type bytes in Python 3. response.content.decode('utf-8') will return a string regardless of Python version so content is now a string. string does not have an attribute of json so that will always raise an exception.

Now the response object has a json method that guesses the encoding and then decodes the json. It looks like this bug was introduced in this commit: [abaa3e558a] and then was worked around but not fixed in this commit: [6a3539882c].

Related to this is the decoding of content that came about because of this ticket: #43. I believe requests now handles this so the decoding can be dropped and response.content.json() can be used.

Here is the code (with unrelated stuff left out): ``` content = response.content.decode('utf-8') # Wrap the json loads in a try, and defer an error # Twitter will return invalid json with an error code in the headers json_error = False try: try: # try to get json content = content.json() except AttributeError: # if unicode detected content = json.loads(content) except ValueError: json_error = True content = {} ``` response is a requests.Response object. response.content is of type str in Python 2.7 and of type bytes in Python 3. `response.content.decode('utf-8')` will return a string regardless of Python version so `content` is now a string. string does not have an attribute of json so that will always raise an exception. Now the response object has a json method that guesses the encoding and then decodes the json. It looks like this bug was introduced in this commit: [abaa3e558a75d225a31d5ad1df93a459361334f9] and then was worked around but not fixed in this commit: [6a3539882caf500e1287c0f71d107a4db1a45862]. Related to this is the decoding of content that came about because of this ticket: #43. I believe requests now handles this so the decoding can be dropped and response.content.json() can be used.

No labels

Bug

Enhancement

Feature Suggestion

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: code/twython#302

No description provided.

Rows
Columns