3.9.1

Fix deprecation warning regarding invalid escape sequences.

.travis.yml

@@ -1,8 +1,13 @@
 language: python
 python:
-  - 2.6
+  - 3.5
-  - 2.7
+  - 3.6
-  - 3.3
+# Enable 3.7 without globally enabling sudo and dist: xenial for other build jobs
+matrix:
+  include:
+    - python: 3.7
+      dist: xenial
+      sudo: true
 env:
   global:
   - secure: USjLDneiXlVvEjkUVqTt+LBi0XJ4QhkRcJzqVXA9gEau1NTjAkNTPmHjUbOygp0dkfoV0uWrZKCw6fL1g+HJgWl0vHeHzcNl4mUkA+OwkGFHgaeIhvUfnyyJA8P3Zm21XHC+ehzMpEFN5fVNNhREjnRj+CXMc0FgA6knwBRobu4=
@@ -17,13 +22,11 @@ env:
   - TEST_LIST_SLUG=team
   - TEST_LIST_OWNER_SCREEN_NAME=twitterapi
   - ACCESS_TOKEN_B64=U2FsdGVkX18QdBhvMNshM4PGy04tU3HLwKP+nNSoNZHKsvGLjELcWEXN2LIu/T+yngX1vGONf9lo14ElnfS4k7sfhiru8phR4+rZuBVP3bDvC2A6fXJuhuLqNhBrWqg32WQewvxLWDWBoKmnvRHg5b74GHh+IN/12tU0cBF2HK8=
-install: pip install -r requirements.txt
+install:
+  - pip install -r requirements.txt
 script: nosetests -v -w tests/ --logging-filter="twython" --with-cov --cov twython --cov-config .coveragerc --cov-report term-missing
 notifications:
   email: false
-branches:
-  only:
-    - master
 after_success:
   - coveralls
 before_script:

AUTHORS.rst

@@ -27,7 +27,7 @@ Patches and Suggestions
 - `Jonathan Elsas <https://github.com/jelsas>`_, Fix for original Streaming API stub causing import errors.
 - `LuqueDaniel <https://github.com/LuqueDaniel>`_, Extended example code where necessary.
 - `Mesar Hameed <https://github.com/mhameed>`_, Commit to swap ``__getattr__`` trick for a more debuggable solution.
-- `Remy DeCausemaker <https://github.com/decause>`_, PEP-8 contributions.
+- `Remy DeCausemaker <https://github.com/decause>`_, PEP 8 contributions.
 - `mckellister <https://github.com/mckellister>`_ Twitter Spring 2012 Clean Up fixes to ``Exception`` raised by Twython (Rate Limits, etc).
 - `Tatz Tsuchiya <http://d.hatena.ne.jp/tatz_tsuchiya/20120115/1326623451>`_, Fix for ``lambda`` scoping in key injection phase.
 - `Mohammed ALDOUB <https://github.com/Voulnet>`_, Fixes for ``http/https`` access endpoints.
@@ -35,7 +35,7 @@ Patches and Suggestions
 - `Terry Jones <https://github.com/terrycojones>`_, Error cleanup and Exception processing in 2.3.0.
 - `Leandro Ferreira <https://github.com/leandroferreira>`_, Fix for double-encoding of search queries in 2.3.0.
 - `Chris Brown <https://github.com/chbrown>`_, Updated to use v1.1 endpoints over v1
-- `Virendra Rajput <https://github.com/bkvirendra>`_, Fixed unicode (json) encoding in twython.py 2.7.2.  
+- `Virendra Rajput <https://github.com/bkvirendra>`_, Fixed unicode (json) encoding in twython.py 2.7.2.
 - `Paul Solbach <https://github.com/hansenrum>`_, fixed requirement for oauth_verifier
 - `Greg Nofi <https://github.com/nofeet>`_, fixed using built-in Exception attributes for storing & retrieving error message
 - `Jonathan Vanasco <https://github.com/jvanasco>`_, Debugging support, error_code tracking, Twitter error API tracking, other fixes
@@ -43,3 +43,23 @@ Patches and Suggestions
 - `Ruben Varela Rosa <https://github.com/rubenvarela>`_, Fixed search example
 - `Oleg Anashkin <https://github.com/extesy>`_, streaming ``handlers`` functionality
 - `Luis Alberto Santana <https://github.com/jackboot7`_, Added auth_endpoint parameter for cases when the right permissions weren't being shown on authenticate step
+- `Cory Benfield <https://github.com/Lukasa>`_, Fixed issue where Twython was unnecessarily disabling compression
+- `Cory Dolphin <https://github.com/corydolphin>`_, Added retry_after attribute to TwythonRateLimitError
+- `Natan L <https://github.com/kuyan>`_, Fixed typo in documentation
+- `Cash Costello <https://github.com/cash>`_, Moved tests to use `responsoes`, fixed typos in documentation
+- `Joe Cabrera <https://github.com/greedo>`_, PEP 8 contributions
+- `bsbkeven <https://github.com/bsbkeven>`_, Added `lookup_status` function to `endpoints.py`
+- `drevicko <https://github.com/drevicko>`_, Added option to yield full page vs individual results in `cursor`
+- `Filipe A Ximenes <https://github.com/filipeximenes>`_, Added `upload_media` function to `endpoints.py`
+- `Mertcan Mermerkaya <https://github.com/mmermerkaya>`_, Fixed code example in documentation
+- `Donne Martin <https://github.com/donnemartin>`_, Fixed typos in `README.rst`
+- `Diego Allen <https://github.com/dalleng>`_, Add missing comma in documentation code snippet
+- `Ben McGinnes <https://github.com/Hasimir>`_, Added mute API endpoints, a couple of examples, random bits.
+- `Davis Silverman <https://github.com/sinistersnare>`_, Fixed endpoint HTTP method
+- `David Beitey <https://github.com/sinistersnare>`_, Fixed documentation typos
+- `7kfpun <https://github.com/7kfpun>`_, Fixed documentation typos
+- `Will Thompson <https://github.com/wjt>`_, Fixed documentation typos
+- `Andre Polykanine <https://github.com/Oire>`_, Fixed documentation typos
+- `Ben Bertka <https://github.com/bbertka>`_, Added dynamic filtering in streamer
+- `Takahashi Shuuji <https://github.com/shuuji3>`_, Fixed documentation typos
+- `Jeremy Keen <https://github.com/jkeenest>`_, Updated documentation example

HISTORY.md

@@ -1,24 +1,61 @@
-.. :changelog:
+# History
 
-History
+## 3.8.0 (2020-04-02)
--------
+- Bump release with latest patches from GitHub.
+- Fix Direct Messages with patches from @manuelcortez.
 
-3.1.2 (2013-12-05)
+## 3.7.0 (2018-07-05)
-++++++++++++++++++
+- Fixes for cursoring API endpoints
+- Improve `html_for_tweet()` parsing
+- Documentation cleanup
+- Documentation for cursor's `return_pages` keyword argument
+- Update links to Twitter API in documentation
+- Added `create_metadata` endpoint
+- Raise error for when cursor is not provided a callable
 
+## 3.6.0 (2017-23-08)
+- Improve replacing of entities with links in `html_for_tweet()`
+- Update classifiers for PyPI
+
+## 3.5.0 (2017-06-06)
+- Added support for "symbols" in `Twython.html_for_tweet()`
+- Added support for extended tweets in `Twython.html_for_tweet()`
+- You can now check progress of video uploads to Twitter when using `Twython.upload_video()`
+
+## 3.4.0 (2016-30-04)
+- Added `upload_video` endpoint
+- Fix quoted status checks in `html_for_tweet`
+- Fix `html_for_tweet` method response when hashtag/mention is a substring of another
+
+## 3.3.0 (2015-18-07)
+- Added support for muting users
+- Fix typos in documentation
+- Updated documentation examples
+- Added dynamic filtering to streamer
+
+## 3.2.0 (2014-10-30)
+- PEP8'd some code
+- Added `lookup_status` function to `endpoints.py`
+- Added keyword argument to `cursor` to return full pages rather than individual results
+- `cursor` now uses while loop rather than recursion
+- Fixed issue where Twython was unnecessarily disabling compression
+- Using `responses` to mock API calls in tests
+- Fixed some typos in  documentation
+- Added `retry_after` attribute to `TwythonRateLimitError`
+- Added `upload_media` method to `Twython` in favor of `update_with_media`
+- Deprecating `update_with_media` per Twitter API 1.1 (https://dev.twitter.com/rest/reference/post/statuses/update_with_media)
+- Unpin `requests` and `requests-oauthlib` in `requirements.txt`
+
+## 3.1.2 (2013-12-05)
 - Fixed Changelog (HISTORY.rst)
 
-3.1.1 (2013-12-05)
+## 3.1.1 (2013-12-05)
-++++++++++++++++++
-
 - Update `requests` version to 2.1.0.
 - Fixed: Streaming issue where `Exceptions` in handlers or `on_success` which subclass `ValueError` would previously be caught and reported as a JSON decoding problem, and `on_error()` would be called (with status_code=200)
 - Fixed issue where XML was returned when bad tokens were passed to `get_authorized_tokens`
 - Fixed import for `setup` causing installation to fail on some devices (eg. Nokia N9/MeeGo)
 
-3.1.0 (2013-09-25)
+## 3.1.0 (2013-09-25)
-++++++++++++++++++
-
 - Added ``html_for_tweet`` static method. This method accepts a tweet object returned from a Twitter API call and will return a string with urls, mentions and hashtags in the tweet replaced with HTML.
 - Pass ``client_args`` to the streaming ``__init__``, much like in core Twython (you can pass headers, timeout, hooks, proxies, etc.).
 - Streamer has new parameter ``handlers`` which accepts a list of strings related to functions that are apart of the Streaming class and start with "on\_". i.e. ['delete'] is passed, when 'delete' is received from a stream response; ``on_delete`` will be called.
@@ -30,9 +67,7 @@ History
 - Fixed streaming issue where results wouldn't be returned for streams that weren't so active (See https://github.com/ryanmcgrath/twython/issues/202#issuecomment-19915708)
 - Streaming API now uses ``_transparent_params`` so when passed ``True`` or ``False`` or an array, etc. Twython formats it to meet Twitter parameter standards (i.e. ['ryanmcgrath', 'mikehelmick', 'twitterapi'] would convert to string 'ryanmcgrath,mikehelmick,twitterapi')
 
-3.0.0 (2013-06-18)
+## 3.0.0 (2013-06-18)
-++++++++++++++++++
-
 - Changed ``twython/twython.py`` to ``twython/api.py`` in attempt to make structure look a little neater
 - Removed all camelCase function access (anything like ``getHomeTimeline`` is now ``get_home_timeline``)
 - Removed ``shorten_url``. With the ``requests`` library, shortening a URL on your own is simple enough
@@ -51,9 +86,7 @@ History
 - Added ``invalidate_token`` API method which allows registed apps to revoke an access token presenting its client credentials
 - ``get_lastfunction_header`` now accepts a ``default_return_value`` parameter. This means that if you pass a second value (ex. ``Twython.get_lastfunction_header('x-rate-limit-remaining', 0)``) and the value is not found, it returns your default value
 
-2.10.1 (2013-05-29)
+## 2.10.1 (2013-05-29)
-++++++++++++++++++
-
 - More test coverage!
 - Fix ``search_gen``
 - Fixed ``get_lastfunction_header`` to actually do what its docstring says, returns ``None`` if header is not found
@@ -63,9 +96,7 @@ History
 - No longer raise ``TwythonStreamError`` when stream line can't be decoded. Instead, sends signal to ``TwythonStreamer.on_error``
 - Allow for (int, long, float) params to be passed to Twython Twitter API functions in Python 2, and (int, float) in Python 3
 
-2.10.0 (2013-05-21)
+## 2.10.0 (2013-05-21)
-++++++++++++++++++
-
 - Added ``get_retweeters_ids`` method
 - Fixed ``TwythonDeprecationWarning`` on camelCase functions if the camelCase was the same as the PEP8 function (i.e. ``Twython.retweet`` did not change)
 - Fixed error message bubbling when error message returned from Twitter was not an array (i.e. if you try to retweet something twice, the error is not found at index 0)
@@ -76,20 +107,14 @@ History
 - Cleaned up ``Twython.construct_api_url``, uses "transparent" parameters (see 4th bullet in this version for explaination)
 - Update ``requests`` and ``requests-oauthlib`` requirements, fixing posting files AND post data together, making authenticated requests in general in Python 3.3
 
-2.9.1 (2013-05-04)
+## 2.9.1 (2013-05-04)
-++++++++++++++++++
-
 - "PEP8" all the functions. Switch functions from camelCase() to underscore_funcs(). (i.e. ``updateStatus()`` is now ``update_status()``)
 
-2.9.0 (2013-05-04)
+## 2.9.0 (2013-05-04)
-++++++++++++++++++
-
 - Fixed streaming issue #144, added ``TwythonStreamer`` to aid users in a friendly streaming experience (streaming examples in ``examples`` and README's have been updated as well)
 - ``Twython`` now requires ``requests-oauthlib`` 0.3.1, fixes #154 (unable to upload media when sending POST data with the file)
 
-2.8.0 (2013-04-29)
+## 2.8.0 (2013-04-29)
-++++++++++++++++++
-
 - Added a ``HISTORY.rst`` to start tracking history of changes
 - Updated ``twitter_endpoints.py`` to ``endpoints.py`` for cleanliness
 - Removed twython3k directory, no longer needed
@@ -112,36 +137,24 @@ History
 - Twython now takes ``ssl_verify`` parameter, defaults True. Set False if you're having development server issues
 - Removed internal ``_media_update`` function, we could have always just used ``self.post``
 
-2.7.3 (2013-04-12)
+## 2.7.3 (2013-04-12)
-++++++++++++++++++
-
 - Fixed issue where Twython Exceptions were not being logged correctly
 
-2.7.2 (2013-04-08)
+## 2.7.2 (2013-04-08)
-++++++++++++++++++
-
 - Fixed ``AttributeError`` when trying to decode the JSON response via ``Response.json()``
 
-2.7.1 (2013-04-08)
+## 2.7.1 (2013-04-08)
-++++++++++++++++++
-
 - Removed ``simplejson`` dependency
 - Fixed ``destroyDirectMessage``, ``createBlock``, ``destroyBlock`` endpoints in ``twitter_endpoints.py``
 - Added ``getProfileBannerSizes`` method to ``twitter_endpoints.py``
 - Made oauth_verifier argument required in ``get_authorized_tokens``
 - Update ``updateProfileBannerImage`` to use v1.1 endpoint
 
-2.7.0 (2013-04-04)
+## 2.7.0 (2013-04-04)
-++++++++++++++++++
-
 - New ``showOwnedLists`` method
 
-2.7.0 (2013-03-31)
+## 2.7.0 (2013-03-31)
-++++++++++++++++++
-
 - Added missing slash to ``getMentionsTimeline`` in ``twitter_endpoints.py``
 
-2.6.0 (2013-03-29)
+## 2.6.0 (2013-03-29)
-++++++++++++++++++
-
 - Updated ``twitter_endpoints.py`` to better reflect order of API endpoints on the Twitter API v1.1 docs site

MANIFEST.in

@@ -1 +1,9 @@
-include README.rst HISTORY.rst LICENSE requirements.txt
+include README.md HISTORY.md LICENSE requirements.txt
+
+recursive-include docs *
+prune docs/_build
+
+recursive-include examples *.py
+
+recursive-include tests *.py
+recursive-include tests/tweets *.json

README.md

@@ -0,0 +1,169 @@
+# Twython
+
+<a href="https://pypi.python.org/pypi/twython"><img src="https://img.shields.io/pypi/v/twython.svg?style=flat-square"></a>
+<a href="https://pypi.python.org/pypi/twython"><img src="https://img.shields.io/pypi/dw/twython.svg?style=flat-square"></a>
+<a href="https://travis-ci.org/ryanmcgrath/twython"><img src="https://img.shields.io/travis/ryanmcgrath/twython.svg?style=flat-square"></a>
+<a href="https://coveralls.io/r/ryanmcgrath/twython?branch=master"><img src="https://img.shields.io/coveralls/ryanmcgrath/twython/master.svg?style=flat-square"></a>
+
+`Twython` is a Python library providing an easy way to access Twitter data. Supports Python 3. It's been battle tested by companies, educational institutions and individuals alike. Try it today!
+
+**Note**: As of Twython 3.7.0, there's a general call for maintainers put out. If you find the project useful and want to help out, reach out to Ryan with the info from the bottom of this README. Great open source project to get your feet wet with!
+
+## Features
+- Query data for:
+    - User information
+    - Twitter lists
+    - Timelines
+    - Direct Messages
+    - and anything found in [the docs](https://developer.twitter.com/en/docs)
+- Image Uploading:
+    - Update user status with an image
+    - Change user avatar
+    - Change user background image
+    - Change user banner image
+- OAuth 2 Application Only (read-only) Support
+- Support for Twitter's Streaming API
+- Seamless Python 3 support!
+
+## Installation
+Install Twython via pip:
+
+```bash
+$ pip install twython
+```
+
+If you're on a legacy project that needs Python 2.7 support, you can install the last version of Twython that supported 2.7:
+
+```
+pip install twython==3.7.0`
+```
+
+Or, if you want the code that is currently on GitHub:
+
+```bash
+git clone git://github.com/ryanmcgrath/twython.git
+cd twython
+python setup.py install
+```
+
+## Documentation
+Documentation is available at https://twython.readthedocs.io/en/latest/
+
+## Starting Out
+First, you'll want to head over to https://apps.twitter.com and register an application!
+
+After you register, grab your applications `Consumer Key` and `Consumer Secret` from the application details tab.
+
+The most common type of authentication is Twitter user authentication using OAuth 1. If you're a web app planning to have users sign up with their Twitter account and interact with their timelines, updating their status, and stuff like that this **is** the authentication for you!
+
+First, you'll want to import Twython
+
+```python
+from twython import Twython
+```
+
+## Obtain Authorization URL
+Now, you'll want to create a Twython instance with your `Consumer Key` and `Consumer Secret`:
+
+- Only pass *callback_url* to *get_authentication_tokens* if your application is a Web Application
+- Desktop and Mobile Applications **do not** require a callback_url
+
+```python
+APP_KEY = 'YOUR_APP_KEY'
+APP_SECRET = 'YOUR_APP_SECRET'
+
+twitter = Twython(APP_KEY, APP_SECRET)
+
+auth = twitter.get_authentication_tokens(callback_url='http://mysite.com/callback')
+```
+
+From the `auth` variable, save the `oauth_token` and `oauth_token_secret` for later use (these are not the final auth tokens). In Django or other web frameworks, you might want to store it to a session variable
+
+```python
+OAUTH_TOKEN = auth['oauth_token']
+OAUTH_TOKEN_SECRET = auth['oauth_token_secret']
+```
+
+Send the user to the authentication url, you can obtain it by accessing
+
+```python
+auth['auth_url']
+```
+
+## Handling the Callback
+If your application is a Desktop or Mobile Application *oauth_verifier* will be the PIN code
+
+After they authorize your application to access some of their account details, they'll be redirected to the callback url you specified in `get_authentication_tokens`.
+
+You'll want to extract the `oauth_verifier` from the url.
+
+Django example:
+
+```python
+oauth_verifier = request.GET['oauth_verifier']
+```
+
+Now that you have the `oauth_verifier` stored to a variable, you'll want to create a new instance of Twython and grab the final user tokens
+
+```python
+twitter = Twython(
+    APP_KEY, APP_SECRET,
+    OAUTH_TOKEN, OAUTH_TOKEN_SECRET
+)
+
+final_step = twitter.get_authorized_tokens(oauth_verifier)
+```
+
+Once you have the final user tokens, store them in a database for later use::
+
+```python
+    OAUTH_TOKEN = final_step['oauth_token']
+    OAUTH_TOKEN_SECRET = final_step['oauth_token_secret']
+```
+
+For OAuth 2 (Application Only, read-only) authentication, see [our documentation](https://twython.readthedocs.io/en/latest/usage/starting_out.html#oauth-2-application-authentication).
+
+## Dynamic Function Arguments
+Keyword arguments to functions are mapped to the functions available for each endpoint in the Twitter API docs. Doing this allows us to be incredibly flexible in querying the Twitter API, so changes to the API aren't held up from you using them by this library.
+
+Basic Usage
+-----------
+
+**Function definitions (i.e. get_home_timeline()) can be found by reading over twython/endpoints.py**
+
+Create a Twython instance with your application keys and the users OAuth tokens
+
+```python
+from twython import Twython
+twitter = Twython(APP_KEY, APP_SECRET, OAUTH_TOKEN, OAUTH_TOKEN_SECRET)
+```
+
+## Authenticated Users Home Timeline
+```python
+twitter.get_home_timeline()
+```
+
+## Updating Status
+This method makes use of dynamic arguments, [read more about them](https://twython.readthedocs.io/en/latest/usage/starting_out.html#dynamic-function-arguments).
+
+```python
+twitter.update_status(status='See how easy using Twython is!')
+```
+
+## Advanced Usage
+- [Advanced Twython Usage](https://twython.readthedocs.io/en/latest/usage/advanced_usage.html)
+- [Streaming with Twython](https://twython.readthedocs.io/en/latest/usage/streaming_api.html)
+
+
+## Questions, Comments, etc?
+My hope is that Twython is so simple that you'd never *have* to ask any questions, but if you feel the need to contact me for this (or other) reasons, you can hit me up at ryan@venodesigns.net.
+
+Or if I'm to busy to answer, feel free to ping mikeh@ydekproductions.com as well.
+
+Follow us on Twitter:
+
+- [@ryanmcgrath](https://twitter.com/ryanmcgrath)
+- [@mikehelmick](https://twitter.com/mikehelmick)
+
+## Want to help?
+Twython is useful, but ultimately only as useful as the people using it (say that ten times fast!). If you'd like to help, write example code, contribute patches, document things on the wiki, tweet about it. Your help is always appreciated!

README.rst

@@ -1,219 +0,0 @@
-Twython
-=======
-
-
-.. image:: https://badge.fury.io/py/twython.png
-        :target: http://badge.fury.io/py/twython
-.. image:: https://travis-ci.org/ryanmcgrath/twython.png?branch=master
-        :target: https://travis-ci.org/ryanmcgrath/twython
-.. image:: https://pypip.in/d/twython/badge.png
-        :target: https://crate.io/packages/twython/
-.. image:: https://coveralls.io/repos/ryanmcgrath/twython/badge.png?branch=master
-        :target: https://coveralls.io/r/ryanmcgrath/twython?branch=master
-
-``Twython`` is the premier Python library providing an easy (and up-to-date) way to access Twitter data. Actively maintained and featuring support for Python 2.6+ and Python 3. It's been battle tested by companies, educational institutions and individuals alike. Try it today!
-
-Features
---------
-
-- Query data for:
-    - User information
-    - Twitter lists
-    - Timelines
-    - Direct Messages
-    - and anything found in `the docs <https://dev.twitter.com/docs/api/1.1>`_
-- Image Uploading:
-    - Update user status with an image
-    - Change user avatar
-    - Change user background image
-    - Change user banner image
-- OAuth 2 Application Only (read-only) Support
-- Support for Twitter's Streaming API
-- Seamless Python 3 support!
-
-Installation
-------------
-
-Install Twython via `pip <http://www.pip-installer.org/>`_
-
-.. code-block:: bash
-
-    $ pip install twython
-
-or, with `easy_install <http://pypi.python.org/pypi/setuptools>`_
-
-.. code-block:: bash
-
-    $ easy_install twython
-
-But, hey... `that's up to you <http://www.pip-installer.org/en/latest/other-tools.html#pip-compared-to-easy-install>`_.
-
-Or, if you want the code that is currently on GitHub
-
-.. code-block:: bash
-
-    git clone git://github.com/ryanmcgrath/twython.git
-    cd twython
-    python setup.py install
-
-Documentation
--------------
-
-Documentation is available at https://twython.readthedocs.org/en/latest/
-
-Starting Out
-------------
-
-First, you'll want to head over to https://dev.twitter.com/apps and register an application!
-
-After you register, grab your applications ``Consumer Key`` and ``Consumer Secret`` from the application details tab.
-
-The most common type of authentication is Twitter user authentication using OAuth 1. If you're a web app planning to have users sign up with their Twitter account and interact with their timelines, updating their status, and stuff like that this **is** the authentication for you!
-
-First, you'll want to import Twython
-
-.. code-block:: python
-
-    from twython import Twython
-
-Authentication
-~~~~~~~~~~~~~~
-
-Obtain Authorization URL
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-Now, you'll want to create a Twython instance with your ``Consumer Key`` and ``Consumer Secret``
-
-    Only pass *callback_url* to *get_authentication_tokens* if your application is a Web Application
-
-    Desktop and Mobile Applications **do not** require a callback_url
-
-.. code-block:: python
-
-    APP_KEY = 'YOUR_APP_KEY'
-    APP_SECRET = 'YOUR_APP_SECRET'
-
-    twitter = Twython(APP_KEY, APP_SECRET)
-
-    auth = twitter.get_authentication_tokens(callback_url='http://mysite.com/callback')
-
-From the ``auth`` variable, save the ``oauth_token`` and ``oauth_token_secret`` for later use (these are not the final auth tokens). In Django or other web frameworks, you might want to store it to a session variable
-
-.. code-block:: python
-
-    OAUTH_TOKEN = auth['oauth_token']
-    OAUTH_TOKEN_SECRET = auth['oauth_token_secret']
-
-Send the user to the authentication url, you can obtain it by accessing
-
-.. code-block:: python
-
-    auth['auth_url']
-
-Handling the Callback
-^^^^^^^^^^^^^^^^^^^^^
-
-    If your application is a Desktop or Mobile Application *oauth_verifier* will be the PIN code
-
-After they authorize your application to access some of their account details, they'll be redirected to the callback url you specified in ``get_autentication_tokens``
-
-You'll want to extract the ``oauth_verifier`` from the url.
-
-Django example:
-
-.. code-block:: python
-
-    oauth_verifier = request.GET['oauth_verifier']
-
-Now that you have the ``oauth_verifier`` stored to a variable, you'll want to create a new instance of Twython and grab the final user tokens
-
-.. code-block:: python
-
-    twitter = Twython(APP_KEY, APP_SECRET,
-                      OAUTH_TOKEN, OAUTH_TOKEN_SECRET)
-
-    final_step = twitter.get_authorized_tokens(oauth_verifier)
-
-Once you have the final user tokens, store them in a database for later use!::
-
-    OAUTH_TOKEN = final_step['oauth_token']
-    OAUTH_TOKEN_SECERT = final_step['oauth_token_secret']
-
-For OAuth 2 (Application Only, read-only) authentication, see `our documentation <https://twython.readthedocs.org/en/latest/usage/starting_out.html#oauth-2-application-authentication>`_
-
-Dynamic Function Arguments
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Keyword arguments to functions are mapped to the functions available for each endpoint in the Twitter API docs. Doing this allows us to be incredibly flexible in querying the Twitter API, so changes to the API aren't held up from you using them by this library.
-
-Basic Usage
------------
-
-**Function definitions (i.e. get_home_timeline()) can be found by reading over twython/endpoints.py**
-
-Create a Twython instance with your application keys and the users OAuth tokens
-
-.. code-block:: python
-
-    from twython import Twython
-    twitter = Twython(APP_KEY, APP_SECRET,
-                      OAUTH_TOKEN, OAUTH_TOKEN_SECRET)
-
-Authenticated Users Home Timeline
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Documentation: https://dev.twitter.com/docs/api/1.1/get/statuses/home_timeline
-
-.. code-block:: python
-
-    twitter.get_home_timeline()
-
-Updating Status
-~~~~~~~~~~~~~~~
-
-This method makes use of dynamic arguments, `read more about them <https://twython.readthedocs.org/en/latest/usage/starting_out.html#dynamic-function-arguments>`_
-
-Documentation: https://dev.twitter.com/docs/api/1/post/statuses/update
-
-.. code-block:: python
-
-    twitter.update_status(status='See how easy using Twython is!')
-
-Searching
-~~~~~~~~~
-
-    https://dev.twitter.com/docs/api/1.1/get/search/tweets says it takes "q" and "result_type" amongst other arguments
-
-.. code-block:: python
-
-    twitter.search(q='twitter')
-    twitter.search(q='twitter', result_type='popular')
-
-Advanced Usage
---------------
-
-- `Advanced Twython Usage <https://twython.readthedocs.org/en/latest/usage/advanced_usage.html>`_
-- `Streaming with Twython <https://twython.readthedocs.org/en/latest/usage/streaming_api.html>`_
-
-
-Notes
------
-
-- Twython 3.0.0 has been injected with 1000mgs of pure awesomeness! OAuth 2 application authentication is now supported. And a *whole lot* more! See the `CHANGELOG <https://github.com/ryanmcgrath/twython/blob/master/HISTORY.rst#300-2013-06-18>`_ for more details!
-
-Questions, Comments, etc?
--------------------------
-
-My hope is that Twython is so simple that you'd never *have* to ask any questions, but if you feel the need to contact me for this (or other) reasons, you can hit me up at ryan@venodesigns.net.
-
-Or if I'm to busy to answer, feel free to ping mikeh@ydekproductions.com as well.
-
-Follow us on Twitter:
-
-- `@ryanmcgrath <https://twitter.com/ryanmcgrath>`_
-- `@mikehelmick <https://twitter.com/mikehelmick>`_
-
-Want to help?
--------------
-
-Twython is useful, but ultimately only as useful as the people using it (say that ten times fast!). If you'd like to help, write example code, contribute patches, document things on the wiki, tweet about it. Your help is always appreciated!

docs/_themes/basicstrap/customsidebar.html

@@ -1,18 +1,5 @@
-<h4>{{ _('Donate') }}</h4>
-
-<p>
-Find Twython useful? Consider supporting the author on <a href="https://www.gittip.com/ryanmcgrath/">Gittip</a>:
-</p>
-
-<p>
-  <iframe style="border: 0; margin: 0; padding: 0;"
-      src="https://www.gittip.com/ryanmcgrath/widget.html"
-      width="48pt" height="20pt"></iframe>
-</p>
-
 <h4>{{ _('Links') }}</h4>
 <ul>
-  <li><a href="https://crate.io/packages/twython/">Twython on Crate</a></li>
   <li><a href="https://github.com/ryanmcgrath/twython">Twython on GitHub</a></li>
   <li><a href="https://github.com/ryanmcgrath/twython/issues">Bug Tracker</a></li>
-</ul>
+</ul>

docs/conf.py

@@ -50,9 +50,9 @@ copyright = u'2013, Ryan McGrath'
 # built documents.
 #
 # The short X.Y version.
-version = '3.1.2'
+version = '3.8.0'
 # The full version, including alpha/beta/rc tags.
-release = '3.1.2'
+release = '3.8.0'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

docs/index.rst

@@ -16,7 +16,7 @@ Features
    - Twitter lists
    - Timelines
    - Direct Messages
-   - and anything found in `the Twitter API docs <https://dev.twitter.com/docs/api/1.1>`_.
+   - and anything found in `the Twitter API docs <https://developer.twitter.com/en/docs>`_.
 - Image Uploading:
    - Update user status with an image
    - Change user avatar

docs/usage/advanced_usage.rst

@@ -16,30 +16,53 @@ Create a Twython instance with your application keys and the users OAuth tokens
 .. code-block:: python
 
     from twython import Twython
-    twitter = Twython(APP_KEY, APP_SECRET
+    twitter = Twython(APP_KEY, APP_SECRET,
                       OAUTH_TOKEN, OAUTH_TOKEN_SECRET)
 
 Updating Status with Image
 --------------------------
 
-Documentation: https://dev.twitter.com/docs/api/1.1/post/statuses/update_with_media
+This uploads an image as a media object and associates it with a status update.
 
 .. code-block:: python
 
     photo = open('/path/to/file/image.jpg', 'rb')
-    twitter.update_status_with_media(status='Checkout this cool image!', media=photo)
+    response = twitter.upload_media(media=photo)
+    twitter.update_status(status='Checkout this cool image!', media_ids=[response['media_id']])
+
+Documentation:
+
+* https://developer.twitter.com/en/docs/api-reference-index
+* https://developer.twitter.com/en/docs/media/upload-media/overview
+
+Updating Status with Video
+--------------------------
+
+This uploads a video as a media object and associates it with a status update.
+
+.. code-block:: python
+
+    video = open('/path/to/file/video.mp4', 'rb')
+    response = twitter.upload_video(media=video, media_type='video/mp4')
+    twitter.update_status(status='Checkout this cool video!', media_ids=[response['media_id']])
+
+Documentation:
+
+* https://developer.twitter.com/en/docs/api-reference-index
+* https://developer.twitter.com/en/docs/media/upload-media/overview
 
 Posting a Status with an Editing Image
 --------------------------------------
 
-    This example resizes an image
+This example resizes an image, then uploads it as a media object and associates it
+with a status update.
 
 .. code-block:: python
 
-    # Assume you are working with a JPEG
+    # Assuming that you are working with a JPEG
 
     from PIL import Image
-    from StringIO import StringIO
+    from io import BytesIO
 
     photo = Image.open('/path/to/file/image.jpg')
 
@@ -48,14 +71,15 @@ Posting a Status with an Editing Image
     height = int((float(photo.size[1]) * float(wpercent)))
     photo = photo.resize((basewidth, height), Image.ANTIALIAS)
 
-    image_io = StringIO.StringIO()
+    image_io = BytesIO()
     photo.save(image_io, format='JPEG')
 
     # If you do not seek(0), the image will be at the end of the file and
     # unable to be read
     image_io.seek(0)
 
-    twitter.update_status_with_media(media=photo, status='Check out my edited image!')
+    response = twitter.upload_media(media=image_io)
+    twitter.update_status(status='Checkout this cool image!', media_ids=[response['media_id']])
 
 
 Search Generator
@@ -67,9 +91,13 @@ That being said, Twython offers a generator for search results and can be access
 
 .. code-block:: python
 
-    search = twitter.search_gen('python')
+    from twython import Twython
-    for result in search:
+    twitter = Twython(APP_KEY, APP_SECRET, OAUTH_TOKEN,
-        print result
+        OAUTH_TOKEN_SECRET)
+
+    results = twitter.cursor(twitter.search, q='python')
+    for result in results:
+        print(result)
 
 Manipulate the Request (headers, proxies, etc.)
 -----------------------------------------------
@@ -90,8 +118,8 @@ Here is an example of sending custom headers to a Twitter API request:
         }
     }
 
-    twitter = Twython(APP_KEY, APP_SECRET
+    twitter = Twython(APP_KEY, APP_SECRET,
-                      OAUTH_TOKEN, OAUTH_TOKEN_SECRET
+                      OAUTH_TOKEN, OAUTH_TOKEN_SECRET,
                       client_args=client_args)
 
 Here is an example of sending the request through proxies:
@@ -107,8 +135,8 @@ Here is an example of sending the request through proxies:
         }
     }
 
-    twitter = Twython(APP_KEY, APP_SECRET
+    twitter = Twython(APP_KEY, APP_SECRET,
-                      OAUTH_TOKEN, OAUTH_TOKEN_SECRET
+                      OAUTH_TOKEN, OAUTH_TOKEN_SECRET,
                       client_args=client_args)
 
 or both (and set a timeout variable):
@@ -128,8 +156,8 @@ or both (and set a timeout variable):
         'timeout': 300,
     }
 
-    twitter = Twython(APP_KEY, APP_SECRET
+    twitter = Twython(APP_KEY, APP_SECRET,
-                      OAUTH_TOKEN, OAUTH_TOKEN_SECRET
+                      OAUTH_TOKEN, OAUTH_TOKEN_SECRET,
                       client_args=client_args)
 
 Access Headers of Previous Call
@@ -142,7 +170,7 @@ If you wish to access headers (ex. x-rate-limit-remaining, x-rate-limit-reset, c
 
     from twython import Twython
 
-    twitter = Twython(APP_KEY, APP_SECRET
+    twitter = Twython(APP_KEY, APP_SECRET,
                       OAUTH_TOKEN, OAUTH_TOKEN_SECRET)
 
     twitter.get_home_timeline()

docs/usage/basic_usage.rst

@@ -22,13 +22,13 @@ Create a Twython instance with your application keys and the users OAuth tokens
 .. code-block:: python
 
     from twython import Twython
-    twitter = Twython(APP_KEY, APP_SECRET
+    twitter = Twython(APP_KEY, APP_SECRET,
                       OAUTH_TOKEN, OAUTH_TOKEN_SECRET)
 
 User Information
 ^^^^^^^^^^^^^^^^
 
-Documentation: https://dev.twitter.com/docs/api/1.1/get/account/verify_credentials
+Documentation: https://developer.twitter.com/en/docs/accounts-and-users/manage-account-settings/api-reference/get-account-verify_credentials
 
 .. code-block:: python
 
@@ -37,7 +37,7 @@ Documentation: https://dev.twitter.com/docs/api/1.1/get/account/verify_credentia
 Authenticated Users Home Timeline
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Documentation: https://dev.twitter.com/docs/api/1.1/get/statuses/home_timeline
+Documentation: https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-home_timeline
 
 .. code-block:: python
 
@@ -48,7 +48,7 @@ Updating Status
 
 This method makes use of dynamic arguments, :ref:`read more about them <dynamicargexplaination>`
 
-Documentation: https://dev.twitter.com/docs/api/1.1/post/statuses/update
+Documentation: https://developer.twitter.com/en/docs/tweets/post-and-engage/api-reference/post-statuses-update
 
 .. code-block:: python
 
@@ -72,7 +72,7 @@ Searching
 
 .. note:: Searching can be done whether you're authenticated via OAuth 1 or OAuth 2
 
-Documentation: https://dev.twitter.com/docs/api/1.1/get/search/tweets
+Documentation: https://developer.twitter.com/en/docs/tweets/search/api-reference/get-search-tweets
 
 .. code-block:: python
 
@@ -80,7 +80,7 @@ Documentation: https://dev.twitter.com/docs/api/1.1/get/search/tweets
 
 .. _dynamicargexplaination:
 
-.. important:: To help explain :ref:`dynamic function arguments <dynamicfunctionarguments>` a little more, you can see that the previous call used the keyword argument ``q``, that is because Twitter specifies in their `search documentation <https://dev.twitter.com/docs/api/1.1/get/search/tweets>`_ that the search call accepts the parameter "q". You can pass mutiple keyword arguments. The search documentation also specifies that the call accepts the parameter "result_type"
+.. important:: To help explain :ref:`dynamic function arguments <dynamicfunctionarguments>` a little more, you can see that the previous call used the keyword argument ``q``, that is because Twitter specifies in their `search documentation <https://developer.twitter.com/en/docs/tweets/search/api-reference/get-search-tweets>`_ that the search call accepts the parameter "q". You can pass mutiple keyword arguments. The search documentation also specifies that the call accepts the parameter "result_type"
 
 .. code-block:: python
 

docs/usage/special_functions.rst

@@ -25,7 +25,7 @@ The Old Way
     results = twitter.search(q='twitter')
     if results.get('statuses'):
         for result in results['statuses']:
-            print result['id_str']
+            print(result['id_str'])
 
 The New Way
 ^^^^^^^^^^^
@@ -37,17 +37,30 @@ The New Way
     twitter = Twython(APP_KEY, APP_SECRET,
                       OAUTH_TOKEN, OAUTH_TOKEN_SECRET)
 
-    results = twitter.cursor(t.search, q='twitter')
+    results = twitter.cursor(twitter.search, q='twitter')
     for result in results:
-        print result['id_str']
+        print(result['id_str'])
 
 Another example:
 
 .. code-block:: python
 
-    results = twitter.cursor(t.get_mentions_timeline)
+    results = twitter.cursor(twitter.get_mentions_timeline)
     for result in results:
-        print result['id_str']
+        print(result['id_str'])
+
+Items vs Pages
+^^^^^^^^^^^^^^
+
+By default, the cursor yields one item at a time. If instead you prefer to work with entire pages of results, specify ``return_pages=True`` as a keyword argument.
+
+.. code-block:: python
+
+    results = twitter.cursor(twitter.get_mentions_timeline, return_pages=True)
+    # page is a list
+    for page in results:
+        for result in page:
+            print(result['id_str'])
 
 
 HTML for Tweet
@@ -66,25 +79,27 @@ This function takes a tweet object received from the Twitter API and returns an
                                             include_rts=True)
     for tweet in user_tweets:
         tweet['text'] = Twython.html_for_tweet(tweet)
-        print tweet['text']
+        print(tweet['text'])
 
 The above code takes all the tweets from a specific users timeline, loops over them and replaces the value of ``tweet['text']`` with HTML.
 
 So:
 
-    http://t.co/FCmXyI6VHd is a #cool site, lol! @mikehelmick should #checkitout. If you can! #thanks Love, @__twython__ https://t.co/67pwRvY6z9
+    http://t.co/FCmXyI6VHd is #cool, lol! @mikehelmick shd #checkitout. Love, @__twython__ $IBM https://t.co/67pwRvY6z9 http://t.co/N6InAO4B71
 
 will be replaced with:
 
-    <a href="http://t.co/FCmXyI6VHd" class="twython-url">google.com</a> is a <a href="https://twitter.com/search?q=%23cool" class="twython-hashtag">#cool</a> site, lol! <a href="https://twitter.com/mikehelmick" class="twython-mention">@mikehelmick</a> should <a href="https://twitter.com/search?q=%23checkitout" class="twython-hashtag">#checkitout</a>. If you can! <a href="https://twitter.com/search?q=%23thanks" class="twython-hashtag">#thanks</a> Love, <a href="https://twitter.com/__twython__" class="twython-mention">@__twython__</a> <a href="https://t.co/67pwRvY6z9" class="twython-url">github.com</a>
+    <a href="http://t.co/FCmXyI6VHd" class="twython-url">google.com</a> is <a href="https://twitter.com/search?q=%23cool" class="twython-hashtag">#cool</a>, lol! <a href="https://twitter.com/mikehelmick" class="twython-mention">@mikehelmick</a> shd <a href="https://twitter.com/search?q=%23checkitout" class="twython-hashtag">#checkitout</a>. Love, <a href="https://twitter.com/__twython__" class="twython-mention">@__twython__</a> <a href="https://twitter.com/?q=%24IBM" class="twython-symbol">$IBM</a> <a href="https://t.co/67pwRvY6z9" class="twython-url">github.com</a> <a href="http://t.co/N6InAO4B71" class="twython-media">pic.twitter.com/N6InAO4B71</a>
 
 .. note:: When converting the string to HTML we add a class to each HTML tag so that you can maninpulate the DOM later on.
 
 - For urls that are replaced we add ``class="twython-url"`` to the anchor tag
+- For media urls that are replaced we add ``class="twython-media"`` to the anchor tag
 - For user mentions that are replaced we add ``class="twython-mention"`` to the anchor tag
 - For hashtags that are replaced we add ``class="twython-hashtag"`` to the anchor tag
+- For symbols that are replaced we add ``class="twython-symbol"`` to the anchor tag
 
-This function excepts two parameters: ``use_display_url`` and ``use_expanded_url``
+This function accepts two parameters: ``use_display_url`` and ``use_expanded_url``
 By default, ``use_display_url`` is ``True``. Meaning the link displayed in the tweet text will appear as (ex. google.com, github.com)
 If ``use_expanded_url`` is ``True``, it overrides ``use_display_url``. The urls will then be displayed as (ex. http://google.com, https://github.com)
 If ``use_display_url`` and ``use_expanded_url`` are ``False``, short url will be used (t.co/xxxxx)

docs/usage/starting_out.rst

@@ -10,7 +10,7 @@ This section is going to help you understand creating a Twitter Application, aut
 Beginning
 ---------
 
-First, you'll want to head over to https://dev.twitter.com/apps and register an application!
+First, you'll want to head over to https://apps.twitter.com/ and register an application!
 
 After you register, grab your applications ``Consumer Key`` and ``Consumer Secret`` from the application details tab.
 
@@ -98,7 +98,7 @@ Once you have the final user tokens, store them in a database for later use!
 .. code-block:: python
 
     OAUTH_TOKEN = final_step['oauth_token']
-    OAUTH_TOKEN_SECERT = final_step['oauth_token_secret']
+    OAUTH_TOKEN_SECRET = final_step['oauth_token_secret']
 
 .. _oauth2:
 

docs/usage/streaming_api.rst

@@ -5,7 +5,7 @@ Streaming API
 
 This section will cover how to use Twython and interact with the Twitter Streaming API.
 
-Streaming Documentation: https://dev.twitter.com/docs/streaming-apis
+Streaming Documentation: https://developer.twitter.com/en/docs/tweets/filter-realtime/guides/streaming-message-types
 
 .. important:: The Streaming API requires that you have OAuth 1 authentication credentials. If you don't have credentials, head over to the :ref:`authentication section <oauth1>` and find out how!
 
@@ -27,10 +27,10 @@ Now set up how you want to handle the signals.
     class MyStreamer(TwythonStreamer):
         def on_success(self, data):
             if 'text' in data:
-                print data['text'].encode('utf-8')
+                print(data['text'])
 
         def on_error(self, status_code, data):
-            print status_code
+            print(status_code)
 
             # Want to stop trying to get data because of the error?
             # Uncomment the next line!

examples/follow_user.py

@@ -0,0 +1,21 @@
+from twython import Twython, TwythonError
+
+# Optionally accept user data from the command line (or elsewhere).
+#
+# Usage:  follow_user.py ryanmcgrath
+
+import sys
+
+if len(sys.argv) >= 2:
+    target = sys.argv[1]
+else:
+    target = raw_input("User to follow: ")
+    # For Python 3.x use: target = input("User to follow: ")
+
+# Requires Authentication as of Twitter API v1.1
+twitter = Twython(APP_KEY, APP_SECRET, OAUTH_TOKEN, OAUTH_TOKEN_SECRET)
+
+try:
+    twitter.create_friendship(screen_name=target, follow="true")
+except TwythonError as e:
+    print(e)

examples/get_direct_messages.py

@@ -0,0 +1,5 @@
+from twython import Twython, TwythonError
+twitter = Twython(APP_KEY, APP_SECRET, OAUTH_TOKEN, OAUTH_TOKEN_SECRET)
+
+get_list = twitter.get_direct_messages()
+print(get_list)

examples/get_user_timeline.py

@@ -7,4 +7,4 @@ try:
 except TwythonError as e:
     print e
 
-print user_timeline
+print(user_timeline)

examples/search_results.py

@@ -8,5 +8,7 @@ except TwythonError as e:
     print e
 
 for tweet in search_results['statuses']:
-    print 'Tweet from @%s Date: %s' % (tweet['user']['screen_name'].encode('utf-8'), tweet['created_at'])
+    print 'Tweet from @%s Date: %s' % (tweet['user']['screen_nam\
+                                       e'].encode('utf-8'),
+                                       tweet['created_at'])
     print tweet['text'].encode('utf-8'), '\n'

examples/stream.py

@@ -16,5 +16,7 @@ stream = MyStreamer(APP_KEY, APP_SECRET,
                     OAUTH_TOKEN, OAUTH_TOKEN_SECRET)
 
 stream.statuses.filter(track='twitter')
-#stream.user()  # Read the authenticated users home timeline (what they see on Twitter) in real-time
+# stream.user()
-#stream.site(follow='twitter')
+# Read the authenticated users home timeline
+# (what they see on Twitter) in real-time
+# stream.site(follow='twitter')

requirements.txt

@@ -1,5 +1,6 @@
 coverage==3.6.0
-requests==2.1.0
+requests>=2.1.0
-requests_oauthlib==0.4.0
+requests_oauthlib>=0.4.0
 python-coveralls==2.1.0
 nose-cov==1.6
+responses==0.3.0

setup.py

@@ -8,8 +8,8 @@ try:
 except ImportError:
     from distutils.core import setup
 
-__author__ = 'Ryan McGrath <ryan@venodesigns.net>'
+__author__ = 'Ryan McGrath <ryan@rymc.io>'
-__version__ = '3.1.2'
+__version__ = '3.9.1'
 
 packages = [
     'twython',
@@ -23,15 +23,16 @@ if sys.argv[-1] == 'publish':
 setup(
     name='twython',
     version=__version__,
-    install_requires=['requests==2.1.0', 'requests_oauthlib==0.4.0'],
+    install_requires=['requests>=2.1.0', 'requests_oauthlib>=0.4.0'],
+    python_requires='>=3.5',
     author='Ryan McGrath',
-    author_email='ryan@venodesigns.net',
+    author_email='ryan@rymc.io',
-    license=open('LICENSE').read(),
+    license='MIT',
     url='https://github.com/ryanmcgrath/twython/tree/master',
     keywords='twitter search api tweet twython stream',
     description='Actively maintained, pure Python wrapper for the Twitter API. Supports both normal and streaming Twitter APIs',
-    long_description=open('README.rst').read() + '\n\n' +
+    long_description=open('README.md', encoding='utf-8').read() + '\n\n' +open('HISTORY.md', encoding='utf-8').read(),
-                     open('HISTORY.rst').read(),
+    long_description_content_type='text/markdown',
     include_package_data=True,
     packages=packages,
     classifiers=[
@@ -40,6 +41,11 @@ setup(
         'License :: OSI Approved :: MIT License',
         'Topic :: Software Development :: Libraries :: Python Modules',
         'Topic :: Communications :: Chat',
-        'Topic :: Internet'
+        'Topic :: Internet',
+        'Programming Language :: Python :: 3',
+        'Programming Language :: Python :: 3 :: Only',
+        'Programming Language :: Python :: 3.5',
+        'Programming Language :: Python :: 3.6',
+        'Programming Language :: Python :: 3.7',
     ]
 )

tests/config.py

@@ -1,5 +1,9 @@
+# -*- coding: utf-8 -*-
 import os
 
+import sys
+import unittest
+
 app_key = os.environ.get('APP_KEY')
 app_secret = os.environ.get('APP_SECRET')
 oauth_token = os.environ.get('OAUTH_TOKEN')
@@ -17,7 +21,6 @@ protected_twitter_2 = os.environ.get('PROTECTED_TWITTER_2', 'TwythonSecure2')
 # Test Ids
 test_tweet_id = os.environ.get('TEST_TWEET_ID', '318577428610031617')
 test_list_slug = os.environ.get('TEST_LIST_SLUG', 'team')
-test_list_owner_screen_name = os.environ.get('TEST_LIST_OWNER_SCREEN_NAME', 'twitterapi')
+test_list_owner_screen_name = os.environ.get('TEST_LIST_OWNER_SCREEN_NAME',
+                                             'twitterapi')
 
-test_tweet_object = {u'contributors': None, u'truncated': False, u'text': u'http://t.co/FCmXyI6VHd is a #cool site, lol! @mikehelmick should #checkitout. If you can! #thanks Love, @__twython__ https://t.co/67pwRvY6z9', u'in_reply_to_status_id': None, u'id': 349683012054683648, u'favorite_count': 0, u'source': u'web', u'retweeted': False, u'coordinates': None, u'entities': {u'symbols': [], u'user_mentions': [{u'id': 29251354, u'indices': [45, 57], u'id_str': u'29251354', u'screen_name': u'mikehelmick', u'name': u'Mike Helmick'}, {u'id': 1431865928, u'indices': [104, 116], u'id_str': u'1431865928', u'screen_name': u'__twython__', u'name': u'Twython'}], u'hashtags': [{u'indices': [28, 33], u'text': u'cool'}, {u'indices': [65, 76], u'text': u'checkitout'}, {u'indices': [90, 97], u'text': u'thanks'}], u'urls': [{u'url': u'http://t.co/FCmXyI6VHd', u'indices': [0, 22], u'expanded_url': u'http://google.com', u'display_url': u'google.com'}, {u'url': u'https://t.co/67pwRvY6z9', u'indices': [117, 140], u'expanded_url': u'https://github.com', u'display_url': u'github.com'}]}, u'in_reply_to_screen_name': None, u'id_str': u'349683012054683648', u'retweet_count': 0, u'in_reply_to_user_id': None, u'favorited': False, u'user': {u'follow_request_sent': False, u'profile_use_background_image': True, u'default_profile_image': True, u'id': 1431865928, u'verified': False, u'profile_text_color': u'333333', u'profile_image_url_https': u'https://si0.twimg.com/sticky/default_profile_images/default_profile_3_normal.png', u'profile_sidebar_fill_color': u'DDEEF6', u'entities': {u'description': {u'urls': []}}, u'followers_count': 1, u'profile_sidebar_border_color': u'C0DEED', u'id_str': u'1431865928', u'profile_background_color': u'3D3D3D', u'listed_count': 0, u'profile_background_image_url_https': u'https://si0.twimg.com/images/themes/theme1/bg.png', u'utc_offset': None, u'statuses_count': 2, u'description': u'', u'friends_count': 1, u'location': u'', u'profile_link_color': u'0084B4', u'profile_image_url': u'http://a0.twimg.com/sticky/default_profile_images/default_profile_3_normal.png', u'following': False, u'geo_enabled': False, u'profile_background_image_url': u'http://a0.twimg.com/images/themes/theme1/bg.png', u'screen_name': u'__twython__', u'lang': u'en', u'profile_background_tile': False, u'favourites_count': 0, u'name': u'Twython', u'notifications': False, u'url': None, u'created_at': u'Thu May 16 01:11:09 +0000 2013', u'contributors_enabled': False, u'time_zone': None, u'protected': False, u'default_profile': False, u'is_translator': False}, u'geo': None, u'in_reply_to_user_id_str': None, u'possibly_sensitive': False, u'lang': u'en', u'created_at': u'Wed Jun 26 00:18:21 +0000 2013', u'in_reply_to_status_id_str': None, u'place': None}
-test_tweet_html = '<a href="http://t.co/FCmXyI6VHd" class="twython-url">google.com</a> is a <a href="https://twitter.com/search?q=%23cool" class="twython-hashtag">#cool</a> site, lol! <a href="https://twitter.com/mikehelmick" class="twython-mention">@mikehelmick</a> should <a href="https://twitter.com/search?q=%23checkitout" class="twython-hashtag">#checkitout</a>. If you can! <a href="https://twitter.com/search?q=%23thanks" class="twython-hashtag">#thanks</a> Love, <a href="https://twitter.com/__twython__" class="twython-mention">@__twython__</a> <a href="https://t.co/67pwRvY6z9" class="twython-url">github.com</a>'

tests/test_auth.py

@@ -1,8 +1,6 @@
 from twython import Twython, TwythonError, TwythonAuthError
 
-from .config import app_key, app_secret, screen_name
+from .config import app_key, app_secret, screen_name, unittest
-
-import unittest
 
 
 class TwythonAuthTestCase(unittest.TestCase):
@@ -16,48 +14,57 @@ class TwythonAuthTestCase(unittest.TestCase):
         self.oauth2_bad_api = Twython('BAD_APP_KEY', 'BAD_APP_SECRET',
                                       oauth_version=2)
 
+    @unittest.skip('skipping non-updated test')
     def test_get_authentication_tokens(self):
         """Test getting authentication tokens works"""
         self.api.get_authentication_tokens(callback_url='http://google.com/',
                                            force_login=True,
                                            screen_name=screen_name)
 
+    @unittest.skip('skipping non-updated test')
     def test_get_authentication_tokens_bad_tokens(self):
         """Test getting authentication tokens with bad tokens
         raises TwythonAuthError"""
         self.assertRaises(TwythonAuthError, self.bad_api.get_authentication_tokens,
                           callback_url='http://google.com/')
 
+    @unittest.skip('skipping non-updated test')
     def test_get_authorized_tokens_bad_tokens(self):
         """Test getting final tokens fails with wrong tokens"""
         self.assertRaises(TwythonError, self.bad_api.get_authorized_tokens,
                           'BAD_OAUTH_VERIFIER')
 
+    @unittest.skip('skipping non-updated test')
     def test_get_authorized_tokens_invalid_or_expired_tokens(self):
         """Test getting final token fails when invalid or expired tokens have been passed"""
         self.assertRaises(TwythonError, self.bad_api_invalid_tokens.get_authorized_tokens,
                          'BAD_OAUTH_VERIFIER')
 
+    @unittest.skip('skipping non-updated test')
     def test_get_authentication_tokens_raises_error_when_oauth2(self):
         """Test when API is set for OAuth 2, get_authentication_tokens raises
         a TwythonError"""
         self.assertRaises(TwythonError, self.oauth2_api.get_authentication_tokens)
 
+    @unittest.skip('skipping non-updated test')
     def test_get_authorization_tokens_raises_error_when_oauth2(self):
         """Test when API is set for OAuth 2, get_authorized_tokens raises
         a TwythonError"""
         self.assertRaises(TwythonError, self.oauth2_api.get_authorized_tokens,
                           'BAD_OAUTH_VERIFIER')
 
+    @unittest.skip('skipping non-updated test')
     def test_obtain_access_token(self):
         """Test obtaining an Application Only OAuth 2 access token succeeds"""
         self.oauth2_api.obtain_access_token()
 
+    @unittest.skip('skipping non-updated test')
     def test_obtain_access_token_bad_tokens(self):
         """Test obtaining an Application Only OAuth 2 access token using bad app tokens fails"""
         self.assertRaises(TwythonAuthError,
                           self.oauth2_bad_api.obtain_access_token)
 
+    @unittest.skip('skipping non-updated test')
     def test_obtain_access_token_raises_error_when_oauth1(self):
         """Test when API is set for OAuth 1, obtain_access_token raises a
         TwythonError"""

tests/test_core.py

@@ -1,503 +1,312 @@
-from twython import Twython, TwythonError, TwythonAuthError
+# -*- coding: utf-8 -*-
+from twython import Twython, TwythonError, TwythonAuthError, TwythonRateLimitError
 
-from .config import (
+from .config import unittest
-    app_key, app_secret, oauth_token, oauth_token_secret,
-    protected_twitter_1, protected_twitter_2, screen_name,
-    test_tweet_id, test_list_slug, test_list_owner_screen_name,
-    access_token, test_tweet_object, test_tweet_html
-)
 
-import time
+import responses
-import unittest
+import requests
+
+from twython.compat import is_py2
+if is_py2:
+    from StringIO import StringIO
+else:
+    from io import StringIO
+
+try:
+    import unittest.mock as mock
+except ImportError:
+    import mock
 
 
 class TwythonAPITestCase(unittest.TestCase):
     def setUp(self):
+        self.api = Twython('', '', '', '')
 
-        client_args = {
+    def get_url(self, endpoint):
-            'headers': {
+        """Convenience function for mapping from endpoint to URL"""
-                'User-Agent': '__twython__ Test'
+        return '%s/%s.json' % (self.api.api_url % self.api.api_version, endpoint)
-            },
-            'allow_redirects': False
-        }
 
-        oauth2_client_args = {
+    def register_response(self, method, url, body='{}', match_querystring=False,
-            'headers': {}  # This is so we can hit coverage that Twython sets User-Agent for us if none is supplied
+                          status=200, adding_headers=None, stream=False,
-        }
+                          content_type='application/json; charset=utf-8'):
+        """Wrapper function for responses for simpler unit tests"""
 
-        self.api = Twython(app_key, app_secret,
+        # responses uses BytesIO to hold the body so it needs to be in bytes
-                           oauth_token, oauth_token_secret,
+        if not is_py2:
-                           client_args=client_args)
+            body = bytes(body, 'UTF-8')
 
-        self.oauth2_api = Twython(app_key, access_token=access_token,
+        responses.add(method, url, body, match_querystring,
-                                  client_args=oauth2_client_args)
+                      status, adding_headers, stream, content_type)
 
+    @responses.activate
+    def test_request_should_handle_full_endpoint(self):
+        """Test that request() accepts a full URL for the endpoint argument"""
+        url = 'https://api.twitter.com/1.1/search/tweets.json'
+        self.register_response(responses.GET, url)
+
+        self.api.request(url)
+
+        self.assertEqual(1, len(responses.calls))
+        self.assertEqual(url, responses.calls[0].request.url)
+
+    @responses.activate
+    def test_request_should_handle_relative_endpoint(self):
+        """Test that request() accepts a twitter endpoint name for the endpoint argument"""
+        url = 'https://api.twitter.com/1.1/search/tweets.json'
+        self.register_response(responses.GET, url)
+
+        self.api.request('search/tweets', version='1.1')
+
+        self.assertEqual(1, len(responses.calls))
+        self.assertEqual(url, responses.calls[0].request.url)
+
+    @responses.activate
+    def test_request_should_post_request_regardless_of_case(self):
+        """Test that request() accepts the HTTP method name regardless of case"""
+        url = 'https://api.twitter.com/1.1/statuses/update.json'
+        self.register_response(responses.POST, url)
+
+        self.api.request(url, method='POST')
+        self.api.request(url, method='post')
+
+        self.assertEqual(2, len(responses.calls))
+        self.assertEqual('POST', responses.calls[0].request.method)
+        self.assertEqual('POST', responses.calls[1].request.method)
+
+    @responses.activate
+    def test_request_should_throw_exception_with_invalid_http_method(self):
+        """Test that request() throws an exception when an invalid HTTP method is passed"""
+        # TODO(cash): should Twython catch the AttributeError and throw a TwythonError
+        self.assertRaises(AttributeError, self.api.request, endpoint='search/tweets', method='INVALID')
+
+    @responses.activate
+    def test_request_should_encode_boolean_as_lowercase_string(self):
+        """Test that request() encodes a boolean parameter as a lowercase string"""
+        endpoint = 'search/tweets'
+        url = self.get_url(endpoint)
+        self.register_response(responses.GET, url)
+
+        self.api.request(endpoint, params={'include_entities': True})
+        self.api.request(endpoint, params={'include_entities': False})
+
+        self.assertEqual(url + '?include_entities=true', responses.calls[0].request.url)
+        self.assertEqual(url + '?include_entities=false', responses.calls[1].request.url)
+
+    @responses.activate
+    def test_request_should_handle_string_or_number_parameter(self):
+        """Test that request() encodes a numeric or string parameter correctly"""
+        endpoint = 'search/tweets'
+        url = self.get_url(endpoint)
+        self.register_response(responses.GET, url)
+
+        self.api.request(endpoint, params={'lang': 'es'})
+        self.api.request(endpoint, params={'count': 50})
+
+        self.assertEqual(url + '?lang=es', responses.calls[0].request.url)
+        self.assertEqual(url + '?count=50', responses.calls[1].request.url)
+
+    @responses.activate
+    def test_request_should_encode_list_of_strings_as_string(self):
+        """Test that request() encodes a list of strings as a comma-separated string"""
+        endpoint = 'search/tweets'
+        url = self.get_url(endpoint)
+        location = ['37.781157', '-122.39872', '1mi']
+        self.register_response(responses.GET, url)
+
+        self.api.request(endpoint, params={'geocode': location})
+
+        # requests url encodes the parameters so , is %2C
+        self.assertEqual(url + '?geocode=37.781157%2C-122.39872%2C1mi', responses.calls[0].request.url)
+
+    @responses.activate
+    def test_request_should_encode_numeric_list_as_string(self):
+        """Test that request() encodes a list of numbers as a comma-separated string"""
+        endpoint = 'search/tweets'
+        url = self.get_url(endpoint)
+        location = [37.781157, -122.39872, '1mi']
+        self.register_response(responses.GET, url)
+
+        self.api.request(endpoint, params={'geocode': location})
+
+        self.assertEqual(url + '?geocode=37.781157%2C-122.39872%2C1mi', responses.calls[0].request.url)
+
+    @responses.activate
+    def test_request_should_ignore_bad_parameter(self):
+        """Test that request() ignores unexpected parameter types"""
+        endpoint = 'search/tweets'
+        url = self.get_url(endpoint)
+        self.register_response(responses.GET, url)
+
+        self.api.request(endpoint, params={'geocode': self})
+
+        self.assertEqual(url, responses.calls[0].request.url)
+
+    @responses.activate
+    def test_request_should_handle_file_as_parameter(self):
+        """Test that request() pulls a file out of params for requests lib"""
+        endpoint = 'account/update_profile_image'
+        url = self.get_url(endpoint)
+        self.register_response(responses.POST, url)
+
+        mock_file = StringIO("Twython test image")
+        self.api.request(endpoint, method='POST', params={'image': mock_file})
+
+        self.assertIn(b'filename="image"', responses.calls[0].request.body)
+        self.assertIn(b"Twython test image", responses.calls[0].request.body)
+
+    @responses.activate
+    def test_request_should_put_params_in_body_when_post(self):
+        """Test that request() passes params as data when the request is a POST"""
+        endpoint = 'statuses/update'
+        url = self.get_url(endpoint)
+        self.register_response(responses.POST, url)
+
+        self.api.request(endpoint, method='POST', params={'status': 'this is a test'})
+
+        self.assertIn(b'status=this+is+a+test', responses.calls[0].request.body)
+        self.assertNotIn('status=this+is+a+test', responses.calls[0].request.url)
+
+    @responses.activate
+    def test_get_uses_get_method(self):
+        """Test Twython generic GET request works"""
+        endpoint = 'account/verify_credentials'
+        url = self.get_url(endpoint)
+        self.register_response(responses.GET, url)
+
+        self.api.get(endpoint)
+
+        self.assertEqual(1, len(responses.calls))
+        self.assertEqual(url, responses.calls[0].request.url)
+
+    @responses.activate
+    def test_post_uses_post_method(self):
+        """Test Twython generic POST request works"""
+        endpoint = 'statuses/update'
+        url = self.get_url(endpoint)
+        self.register_response(responses.POST, url)
+
+        self.api.post(endpoint, params={'status': 'I love Twython!'})
+
+        self.assertEqual(1, len(responses.calls))
+        self.assertEqual(url, responses.calls[0].request.url)
+
+    def test_raise_twython_error_on_request_exception(self):
+        """Test if TwythonError is raised by a RequestException"""
+        with mock.patch.object(requests.Session, 'get') as get_mock:
+            # mocking an ssl cert error
+            get_mock.side_effect = requests.RequestException("hostname 'example.com' doesn't match ...")
+            self.assertRaises(TwythonError, self.api.get, 'https://example.com')
+
+    @responses.activate
+    def test_request_should_get_convert_json_to_data(self):
+        """Test that Twython converts JSON data to a Python object"""
+        endpoint = 'statuses/show'
+        url = self.get_url(endpoint)
+        self.register_response(responses.GET, url, body='{"id": 210462857140252672}')
+
+        data = self.api.request(endpoint, params={'id': 210462857140252672})
+
+        self.assertEqual({'id': 210462857140252672}, data)
+
+    @responses.activate
+    def test_request_should_raise_exception_with_invalid_json(self):
+        """Test that Twython handles invalid JSON (though Twitter should not return it)"""
+        endpoint = 'statuses/show'
+        url = self.get_url(endpoint)
+        self.register_response(responses.GET, url, body='{"id: 210462857140252672}')
+
+        self.assertRaises(TwythonError, self.api.request, endpoint, params={'id': 210462857140252672})
+
+    @responses.activate
+    def test_request_should_handle_401(self):
+        """Test that Twython raises an auth error on 401 error"""
+        endpoint = 'statuses/home_timeline'
+        url = self.get_url(endpoint)
+        self.register_response(responses.GET, url, body='{"errors":[{"message":"Error"}]}', status=401)
+
+        self.assertRaises(TwythonAuthError, self.api.request, endpoint)
+
+    @responses.activate
+    def test_request_should_handle_400_for_missing_auth_data(self):
+        """Test that Twython raises an auth error on 400 error when no oauth data sent"""
+        endpoint = 'statuses/home_timeline'
+        url = self.get_url(endpoint)
+        self.register_response(responses.GET, url,
+                               body='{"errors":[{"message":"Bad Authentication data"}]}', status=400)
+
+        self.assertRaises(TwythonAuthError, self.api.request, endpoint)
+
+    @responses.activate
+    def test_request_should_handle_400_that_is_not_auth_related(self):
+        """Test that Twython raises a normal error on 400 error when unrelated to authorization"""
+        endpoint = 'statuses/home_timeline'
+        url = self.get_url(endpoint)
+        self.register_response(responses.GET, url,
+                               body='{"errors":[{"message":"Bad request"}]}', status=400)
+
+        self.assertRaises(TwythonError, self.api.request, endpoint)
+
+    @responses.activate
+    def test_request_should_handle_rate_limit(self):
+        """Test that Twython raises an rate limit error on 429"""
+        endpoint = 'statuses/home_timeline'
+        url = self.get_url(endpoint)
+        self.register_response(responses.GET, url,
+                               body='{"errors":[{"message":"Rate Limit"}]}', status=429)
+
+        self.assertRaises(TwythonRateLimitError, self.api.request, endpoint)
+
+    @responses.activate
+    def test_get_lastfunction_header_should_return_header(self):
+        """Test getting last specific header of the last API call works"""
+        endpoint = 'statuses/home_timeline'
+        url = self.get_url(endpoint)
+        self.register_response(responses.GET, url, adding_headers={'x-rate-limit-remaining': '37'})
+
+        self.api.get(endpoint)
+
+        value = self.api.get_lastfunction_header('x-rate-limit-remaining')
+        self.assertEqual('37', value)
+        value2 = self.api.get_lastfunction_header('does-not-exist')
+        self.assertIsNone(value2)
+        value3 = self.api.get_lastfunction_header('not-there-either', '96')
+        self.assertEqual('96', value3)
+
+    def test_get_lastfunction_header_should_raise_error_when_no_previous_call(self):
+        """Test attempting to get a header when no API call was made raises a TwythonError"""
+        self.assertRaises(TwythonError, self.api.get_lastfunction_header, 'no-api-call-was-made')
+
+    @responses.activate
+    def test_sends_correct_accept_encoding_header(self):
+        """Test that Twython accepts compressed data."""
+        endpoint = 'statuses/home_timeline'
+        url = self.get_url(endpoint)
+        self.register_response(responses.GET, url)
+
+        self.api.get(endpoint)
+
+        self.assertEqual(b'gzip, deflate', responses.calls[0].request.headers['Accept-Encoding'])
+
+    # Static methods
     def test_construct_api_url(self):
         """Test constructing a Twitter API url works as we expect"""
         url = 'https://api.twitter.com/1.1/search/tweets.json'
         constructed_url = self.api.construct_api_url(url, q='#twitter')
         self.assertEqual(constructed_url, 'https://api.twitter.com/1.1/search/tweets.json?q=%23twitter')
 
-    def test_get(self):
-        """Test Twython generic GET request works"""
-        self.api.get('account/verify_credentials')
-
-    def test_post(self):
-        """Test Twython generic POST request works, with a full url and
-        with just an endpoint"""
-        update_url = 'https://api.twitter.com/1.1/statuses/update.json'
-        status = self.api.post(update_url, params={'status': 'I love Twython! %s' % int(time.time())})
-        self.api.post('statuses/destroy/%s' % status['id_str'])
-
-    def test_get_lastfunction_header(self):
-        """Test getting last specific header of the last API call works"""
-        self.api.get('statuses/home_timeline')
-        self.api.get_lastfunction_header('x-rate-limit-remaining')
-
-    def test_get_lastfunction_header_not_present(self):
-        """Test getting specific header that does not exist from the last call returns None"""
-        self.api.get('statuses/home_timeline')
-        header = self.api.get_lastfunction_header('does-not-exist')
-        self.assertEqual(header, None)
-
-    def test_get_lastfunction_header_no_last_api_call(self):
-        """Test attempting to get a header when no API call was made raises a TwythonError"""
-        self.assertRaises(TwythonError, self.api.get_lastfunction_header,
-                          'no-api-call-was-made')
-
-    def test_cursor(self):
-        """Test looping through the generator results works, at least once that is"""
-        search = self.api.cursor(self.api.search, q='twitter', count=1)
-        counter = 0
-        while counter < 2:
-            counter += 1
-            result = next(search)
-            new_id_str = int(result['id_str'])
-            if counter == 1:
-                prev_id_str = new_id_str
-                time.sleep(1)  # Give time for another tweet to come into search
-            if counter == 2:
-                self.assertTrue(new_id_str > prev_id_str)
-
     def test_encode(self):
         """Test encoding UTF-8 works"""
         self.api.encode('Twython is awesome!')
 
-    def test_html_for_tweet(self):
+    def test_cursor_requires_twython_function(self):
-        """Test HTML for Tweet returns what we want"""
+        """Test that cursor() raises when called without a Twython function"""
-        tweet_text = self.api.html_for_tweet(test_tweet_object)
+        def init_and_iterate_cursor(*args, **kwargs):
-        self.assertEqual(test_tweet_html, tweet_text)
+            cursor = self.api.cursor(*args, **kwargs)
+            return next(cursor)
 
-    def test_html_for_tweet_expanded_url(self):
+        non_function = object()
-        """Test using expanded url in HTML for Tweet displays full urls"""
+        non_twython_function = lambda x: x
-        tweet_text = self.api.html_for_tweet(test_tweet_object,
-                                             use_expanded_url=True)
-        # Make sure full url is in HTML
-        self.assertTrue('http://google.com' in tweet_text)
 
-    def test_html_for_tweet_short_url(self):
+        self.assertRaises(TypeError, init_and_iterate_cursor, non_function)
-        """Test using expanded url in HTML for Tweet displays full urls"""
+        self.assertRaises(TwythonError, init_and_iterate_cursor, non_twython_function)
-        tweet_text = self.api.html_for_tweet(test_tweet_object, False)
-        # Make sure HTML doesn't contain the display OR exapanded url
-        self.assertTrue(not 'http://google.com' in tweet_text)
-        self.assertTrue(not 'google.com' in tweet_text)
 
-    def test_raise_error_on_bad_ssl_cert(self):
-        """Test TwythonError is raised by a RequestException when an actual HTTP happens"""
-        self.assertRaises(TwythonError, self.api.get, 'https://example.com')
-
-    # Timelines
-    def test_get_mentions_timeline(self):
-        """Test returning mentions timeline for authenticated user succeeds"""
-        self.api.get_mentions_timeline()
-
-    def test_get_user_timeline(self):
-        """Test returning timeline for authenticated user and random user
-        succeeds"""
-        self.api.get_user_timeline()  # Authenticated User Timeline
-        self.api.get_user_timeline(screen_name='twitter')  # Random User Timeline
-
-    def test_get_protected_user_timeline_following(self):
-        """Test returning a protected user timeline who you are following
-        succeeds"""
-        self.api.get_user_timeline(screen_name=protected_twitter_1)
-
-    def test_get_protected_user_timeline_not_following(self):
-        """Test returning a protected user timeline who you are not following
-        fails and raise a TwythonAuthError"""
-        self.assertRaises(TwythonAuthError, self.api.get_user_timeline,
-                          screen_name=protected_twitter_2)
-
-    def test_retweeted_of_me(self):
-        """Test that getting recent tweets by authenticated user that have
-        been retweeted by others succeeds"""
-        self.api.retweeted_of_me()
-
-    def test_get_home_timeline(self):
-        """Test returning home timeline for authenticated user succeeds"""
-        self.api.get_home_timeline()
-
-    # Tweets
-    def test_get_retweets(self):
-        """Test getting retweets of a specific tweet succeeds"""
-        self.api.get_retweets(id=test_tweet_id)
-
-    def test_show_status(self):
-        """Test returning a single status details succeeds"""
-        self.api.show_status(id=test_tweet_id)
-
-    def test_update_and_destroy_status(self):
-        """Test updating and deleting a status succeeds"""
-        status = self.api.update_status(status='Test post just to get deleted :( %s' % int(time.time()))
-        self.api.destroy_status(id=status['id_str'])
-
-    def test_get_oembed_tweet(self):
-        """Test getting info to embed tweet on Third Party site succeeds"""
-        self.api.get_oembed_tweet(id='99530515043983360')
-
-    def test_get_retweeters_ids(self):
-        """Test getting ids for people who retweeted a tweet succeeds"""
-        self.api.get_retweeters_ids(id='99530515043983360')
-
-    # Search
-    def test_search(self):
-        """Test searching tweets succeeds"""
-        self.api.search(q='twitter')
-
-    # Direct Messages
-    def test_get_direct_messages(self):
-        """Test getting the authenticated users direct messages succeeds"""
-        self.api.get_direct_messages()
-
-    def test_get_sent_messages(self):
-        """Test getting the authenticated users direct messages they've
-        sent succeeds"""
-        self.api.get_sent_messages()
-
-    def test_send_get_and_destroy_direct_message(self):
-        """Test sending, getting, then destory a direct message succeeds"""
-        message = self.api.send_direct_message(screen_name=protected_twitter_1,
-                                               text='Hey d00d! %s' % int(time.time()))
-
-        self.api.get_direct_message(id=message['id_str'])
-        self.api.destroy_direct_message(id=message['id_str'])
-
-    def test_send_direct_message_to_non_follower(self):
-        """Test sending a direct message to someone who doesn't follow you
-        fails"""
-        self.assertRaises(TwythonError, self.api.send_direct_message,
-                          screen_name=protected_twitter_2, text='Yo, man! %s' % int(time.time()))
-
-    # Friends & Followers
-    def test_get_user_ids_of_blocked_retweets(self):
-        """Test that collection of user_ids that the authenticated user does
-        not want to receive retweets from succeeds"""
-        self.api.get_user_ids_of_blocked_retweets(stringify_ids=True)
-
-    def test_get_friends_ids(self):
-        """Test returning ids of users the authenticated user and then a random
-        user is following succeeds"""
-        self.api.get_friends_ids()
-        self.api.get_friends_ids(screen_name='twitter')
-
-    def test_get_followers_ids(self):
-        """Test returning ids of users the authenticated user and then a random
-        user are followed by succeeds"""
-        self.api.get_followers_ids()
-        self.api.get_followers_ids(screen_name='twitter')
-
-    def test_lookup_friendships(self):
-        """Test returning relationships of the authenticating user to the
-        comma-separated list of up to 100 screen_names or user_ids provided
-        succeeds"""
-        self.api.lookup_friendships(screen_name='twitter,ryanmcgrath')
-
-    def test_get_incoming_friendship_ids(self):
-        """Test returning incoming friendship ids succeeds"""
-        self.api.get_incoming_friendship_ids()
-
-    def test_get_outgoing_friendship_ids(self):
-        """Test returning outgoing friendship ids succeeds"""
-        self.api.get_outgoing_friendship_ids()
-
-    def test_create_friendship(self):
-        """Test creating a friendship succeeds"""
-        self.api.create_friendship(screen_name='justinbieber')
-
-    def test_destroy_friendship(self):
-        """Test destroying a friendship succeeds"""
-        self.api.destroy_friendship(screen_name='justinbieber')
-
-    def test_update_friendship(self):
-        """Test updating friendships succeeds"""
-        self.api.update_friendship(screen_name=protected_twitter_1,
-                                   retweets='true')
-
-        self.api.update_friendship(screen_name=protected_twitter_1,
-                                   retweets=False)
-
-    def test_show_friendships(self):
-        """Test showing specific friendship succeeds"""
-        self.api.show_friendship(target_screen_name=protected_twitter_1)
-
-    def test_get_friends_list(self):
-        """Test getting list of users authenticated user then random user is
-        following succeeds"""
-        self.api.get_friends_list()
-        self.api.get_friends_list(screen_name='twitter')
-
-    def test_get_followers_list(self):
-        """Test getting list of users authenticated user then random user are
-        followed by succeeds"""
-        self.api.get_followers_list()
-        self.api.get_followers_list(screen_name='twitter')
-
-    # Users
-    def test_get_account_settings(self):
-        """Test getting the authenticated user account settings succeeds"""
-        self.api.get_account_settings()
-
-    def test_verify_credentials(self):
-        """Test representation of the authenticated user call succeeds"""
-        self.api.verify_credentials()
-
-    def test_update_account_settings(self):
-        """Test updating a user account settings succeeds"""
-        self.api.update_account_settings(lang='en')
-
-    def test_update_delivery_service(self):
-        """Test updating delivery settings fails because we don't have
-        a mobile number on the account"""
-        self.assertRaises(TwythonError, self.api.update_delivery_service,
-                          device='none')
-
-    def test_update_profile(self):
-        """Test updating profile succeeds"""
-        self.api.update_profile(include_entities='true')
-
-    def test_update_profile_colors(self):
-        """Test updating profile colors succeeds"""
-        self.api.update_profile_colors(profile_background_color='3D3D3D')
-
-    def test_list_blocks(self):
-        """Test listing users who are blocked by the authenticated user
-        succeeds"""
-        self.api.list_blocks()
-
-    def test_list_block_ids(self):
-        """Test listing user ids who are blocked by the authenticated user
-        succeeds"""
-        self.api.list_block_ids()
-
-    def test_create_block(self):
-        """Test blocking a user succeeds"""
-        self.api.create_block(screen_name='justinbieber')
-
-    def test_destroy_block(self):
-        """Test unblocking a user succeeds"""
-        self.api.destroy_block(screen_name='justinbieber')
-
-    def test_lookup_user(self):
-        """Test listing a number of user objects succeeds"""
-        self.api.lookup_user(screen_name='twitter,justinbieber')
-
-    def test_show_user(self):
-        """Test showing one user works"""
-        self.api.show_user(screen_name='twitter')
-
-    def test_search_users(self):
-        """Test that searching for users succeeds"""
-        self.api.search_users(q='Twitter API')
-
-    def test_get_contributees(self):
-        """Test returning list of accounts the specified user can
-        contribute to succeeds"""
-        self.api.get_contributees(screen_name='TechCrunch')
-
-    def test_get_contributors(self):
-        """Test returning list of accounts that contribute to the
-        authenticated user fails because we are not a Contributor account"""
-        self.assertRaises(TwythonError, self.api.get_contributors,
-                          screen_name=screen_name)
-
-    def test_remove_profile_banner(self):
-        """Test removing profile banner succeeds"""
-        self.api.remove_profile_banner()
-
-    def test_get_profile_banner_sizes(self):
-        """Test getting list of profile banner sizes fails because
-        we have not uploaded a profile banner"""
-        self.assertRaises(TwythonError, self.api.get_profile_banner_sizes)
-
-    # Suggested Users
-    def test_get_user_suggestions_by_slug(self):
-        """Test getting user suggestions by slug succeeds"""
-        self.api.get_user_suggestions_by_slug(slug='twitter')
-
-    def test_get_user_suggestions(self):
-        """Test getting user suggestions succeeds"""
-        self.api.get_user_suggestions()
-
-    def test_get_user_suggestions_statuses_by_slug(self):
-        """Test getting status of suggested users succeeds"""
-        self.api.get_user_suggestions_statuses_by_slug(slug='funny')
-
-    # Favorites
-    def test_get_favorites(self):
-        """Test getting list of favorites for the authenticated
-        user succeeds"""
-        self.api.get_favorites()
-
-    def test_create_and_destroy_favorite(self):
-        """Test creating and destroying a favorite on a tweet succeeds"""
-        self.api.create_favorite(id=test_tweet_id)
-        self.api.destroy_favorite(id=test_tweet_id)
-
-    # Lists
-    def test_show_lists(self):
-        """Test show lists for specified user"""
-        self.api.show_lists(screen_name='twitter')
-
-    def test_get_list_statuses(self):
-        """Test timeline of tweets authored by members of the
-        specified list succeeds"""
-        self.api.get_list_statuses(slug=test_list_slug,
-                                   owner_screen_name=test_list_owner_screen_name)
-
-    def test_create_update_destroy_list_add_remove_list_members(self):
-        """Test create a list, adding and removing members then
-        deleting the list succeeds"""
-        the_list = self.api.create_list(name='Stuff %s' % int(time.time()))
-        list_id = the_list['id_str']
-
-        self.api.update_list(list_id=list_id, name='Stuff Renamed %s' % int(time.time()))
-
-        screen_names = ['johncena', 'xbox']
-        # Multi add/delete members
-        self.api.create_list_members(list_id=list_id,
-                                     screen_name=screen_names)
-        self.api.delete_list_members(list_id=list_id,
-                                     screen_name=screen_names)
-
-        # Single add/delete member
-        self.api.add_list_member(list_id=list_id, screen_name='justinbieber')
-        self.api.delete_list_member(list_id=list_id, screen_name='justinbieber')
-
-        self.api.delete_list(list_id=list_id)
-
-    def test_get_list_memberships(self):
-        """Test list of memberhips the authenticated user succeeds"""
-        self.api.get_list_memberships()
-
-    def test_get_list_subscribers(self):
-        """Test list of subscribers of a specific list succeeds"""
-        self.api.get_list_subscribers(slug=test_list_slug,
-                                      owner_screen_name=test_list_owner_screen_name)
-
-    def test_subscribe_is_subbed_and_unsubscribe_to_list(self):
-        """Test subscribing, is a list sub and unsubbing to list succeeds"""
-        self.api.subscribe_to_list(slug=test_list_slug,
-                                   owner_screen_name=test_list_owner_screen_name)
-        # Returns 404 if user is not a subscriber
-        self.api.is_list_subscriber(slug=test_list_slug,
-                                    owner_screen_name=test_list_owner_screen_name,
-                                    screen_name=screen_name)
-        self.api.unsubscribe_from_list(slug=test_list_slug,
-                                       owner_screen_name=test_list_owner_screen_name)
-
-    def test_is_list_member(self):
-        """Test returning if specified user is member of a list succeeds"""
-        # Returns 404 if not list member
-        self.api.is_list_member(slug=test_list_slug,
-                                owner_screen_name=test_list_owner_screen_name,
-                                screen_name='themattharris')
-
-    def test_get_list_members(self):
-        """Test listing members of the specified list succeeds"""
-        self.api.get_list_members(slug=test_list_slug,
-                                  owner_screen_name=test_list_owner_screen_name)
-
-    def test_get_specific_list(self):
-        """Test getting specific list succeeds"""
-        self.api.get_specific_list(slug=test_list_slug,
-                                   owner_screen_name=test_list_owner_screen_name)
-
-    def test_get_list_subscriptions(self):
-        """Test collection of the lists the specified user is
-        subscribed to succeeds"""
-        self.api.get_list_subscriptions(screen_name='twitter')
-
-    def test_show_owned_lists(self):
-        """Test collection of lists the specified user owns succeeds"""
-        self.api.show_owned_lists(screen_name='twitter')
-
-    # Saved Searches
-    def test_get_saved_searches(self):
-        """Test getting list of saved searches for authenticated
-        user succeeds"""
-        self.api.get_saved_searches()
-
-    def test_create_get_destroy_saved_search(self):
-        """Test getting list of saved searches for authenticated
-        user succeeds"""
-        saved_search = self.api.create_saved_search(query='#Twitter')
-        saved_search_id = saved_search['id_str']
-
-        self.api.show_saved_search(id=saved_search_id)
-        self.api.destroy_saved_search(id=saved_search_id)
-
-    # Places & Geo
-    def test_get_geo_info(self):
-        """Test getting info about a geo location succeeds"""
-        self.api.get_geo_info(place_id='df51dec6f4ee2b2c')
-
-    def test_reverse_geo_code(self):
-        """Test reversing geocode succeeds"""
-        self.api.reverse_geocode(lat='37.76893497', long='-122.42284884')
-
-    def test_search_geo(self):
-        """Test search for places that can be attached
-        to a statuses/update succeeds"""
-        self.api.search_geo(query='Toronto')
-
-    def test_get_similar_places(self):
-        """Test locates places near the given coordinates which
-        are similar in name succeeds"""
-        self.api.get_similar_places(lat='37', long='-122', name='Twitter HQ')
-
-    # Trends
-    def test_get_place_trends(self):
-        """Test getting the top 10 trending topics for a specific
-        WOEID succeeds"""
-        self.api.get_place_trends(id=1)
-
-    def test_get_available_trends(self):
-        """Test returning locations that Twitter has trending
-        topic information for succeeds"""
-        self.api.get_available_trends()
-
-    def test_get_closest_trends(self):
-        """Test getting the locations that Twitter has trending topic
-        information for, closest to a specified location succeeds"""
-        self.api.get_closest_trends(lat='37', long='-122')
-
-    # Help
-    def test_get_twitter_configuration(self):
-        """Test getting Twitter's configuration succeeds"""
-        self.api.get_twitter_configuration()
-
-    def test_get_supported_languages(self):
-        """Test getting languages supported by Twitter succeeds"""
-        self.api.get_supported_languages()
-
-    def test_privacy_policy(self):
-        """Test getting Twitter's Privacy Policy succeeds"""
-        self.api.get_privacy_policy()
-
-    def test_get_tos(self):
-        """Test getting the Twitter Terms of Service succeeds"""
-        self.api.get_tos()
-
-    def test_get_application_rate_limit_status(self):
-        """Test getting application rate limit status succeeds"""
-        self.oauth2_api.get_application_rate_limit_status()

tests/test_endpoints.py

@@ -0,0 +1,533 @@
+from twython import Twython, TwythonError, TwythonAuthError
+
+from .config import (
+    app_key, app_secret, oauth_token, oauth_token_secret,
+    protected_twitter_1, protected_twitter_2, screen_name,
+    test_tweet_id, test_list_slug, test_list_owner_screen_name,
+    access_token, unittest
+)
+
+import time
+
+
+class TwythonEndpointsTestCase(unittest.TestCase):
+    def setUp(self):
+
+        client_args = {
+            'headers': {
+                'User-Agent': '__twython__ Test'
+            },
+            'allow_redirects': False
+        }
+
+        # This is so we can hit coverage that Twython sets
+        # User-Agent for us if none is supplied
+        oauth2_client_args = {
+            'headers': {}
+        }
+
+        self.api = Twython(app_key, app_secret,
+                           oauth_token, oauth_token_secret,
+                           client_args=client_args)
+
+        self.oauth2_api = Twython(app_key, access_token=access_token,
+                                  client_args=oauth2_client_args)
+
+    # Timelines
+    @unittest.skip('skipping non-updated test')
+    def test_get_mentions_timeline(self):
+        """Test returning mentions timeline for authenticated user succeeds"""
+        self.api.get_mentions_timeline()
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_user_timeline(self):
+        """Test returning timeline for authenticated user and random user
+        succeeds"""
+        self.api.get_user_timeline()  # Authenticated User Timeline
+        self.api.get_user_timeline(screen_name='twitter')
+        # Random User Timeline
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_protected_user_timeline_following(self):
+        """Test returning a protected user timeline who you are following
+        succeeds"""
+        self.api.get_user_timeline(screen_name=protected_twitter_1)
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_protected_user_timeline_not_following(self):
+        """Test returning a protected user timeline who you are not following
+        fails and raise a TwythonAuthError"""
+        self.assertRaises(TwythonAuthError, self.api.get_user_timeline,
+                          screen_name=protected_twitter_2)
+
+    @unittest.skip('skipping non-updated test')
+    def test_retweeted_of_me(self):
+        """Test that getting recent tweets by authenticated user that have
+        been retweeted by others succeeds"""
+        self.api.retweeted_of_me()
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_home_timeline(self):
+        """Test returning home timeline for authenticated user succeeds"""
+        self.api.get_home_timeline()
+
+    # Tweets
+    @unittest.skip('skipping non-updated test')
+    def test_get_retweets(self):
+        """Test getting retweets of a specific tweet succeeds"""
+        self.api.get_retweets(id=test_tweet_id)
+
+    @unittest.skip('skipping non-updated test')
+    def test_show_status(self):
+        """Test returning a single status details succeeds"""
+        self.api.show_status(id=test_tweet_id)
+
+    @unittest.skip('skipping non-updated test')
+    def test_update_and_destroy_status(self):
+        """Test updating and deleting a status succeeds"""
+        status = self.api.update_status(status='Test post just to get \
+                    deleted :( %s' % int(time.time()))
+        self.api.destroy_status(id=status['id_str'])
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_oembed_tweet(self):
+        """Test getting info to embed tweet on Third Party site succeeds"""
+        self.api.get_oembed_tweet(id='99530515043983360')
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_retweeters_ids(self):
+        """Test getting ids for people who retweeted a tweet succeeds"""
+        self.api.get_retweeters_ids(id='99530515043983360')
+
+    # Search
+    @unittest.skip('skipping non-updated test')
+    def test_search(self):
+        """Test searching tweets succeeds"""
+        self.api.search(q='twitter')
+
+    # Direct Messages
+    @unittest.skip('skipping non-updated test')
+    def test_get_direct_messages(self):
+        """Test getting the authenticated users direct messages succeeds"""
+        self.api.get_direct_messages()
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_sent_messages(self):
+        """Test getting the authenticated users direct messages they've
+        sent succeeds"""
+        self.api.get_sent_messages()
+
+    @unittest.skip('skipping non-updated test')
+    def test_send_get_and_destroy_direct_message(self):
+        """Test sending, getting, then destory a direct message succeeds"""
+        message = self.api.send_direct_message(screen_name=protected_twitter_1,
+                                               text='Hey d00d! %s\
+                                               ' % int(time.time()))
+
+        self.api.get_direct_message(id=message['id_str'])
+        self.api.destroy_direct_message(id=message['id_str'])
+
+    @unittest.skip('skipping non-updated test')
+    def test_send_direct_message_to_non_follower(self):
+        """Test sending a direct message to someone who doesn't follow you
+        fails"""
+        self.assertRaises(TwythonError, self.api.send_direct_message,
+                          screen_name=protected_twitter_2, text='Yo, man! \
+                          %s' % int(time.time()))
+
+    # Friends & Followers
+    @unittest.skip('skipping non-updated test')
+    def test_get_user_ids_of_blocked_retweets(self):
+        """Test that collection of user_ids that the authenticated user does
+        not want to receive retweets from succeeds"""
+        self.api.get_user_ids_of_blocked_retweets(stringify_ids=True)
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_friends_ids(self):
+        """Test returning ids of users the authenticated user and then a random
+        user is following succeeds"""
+        self.api.get_friends_ids()
+        self.api.get_friends_ids(screen_name='twitter')
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_followers_ids(self):
+        """Test returning ids of users the authenticated user and then a random
+        user are followed by succeeds"""
+        self.api.get_followers_ids()
+        self.api.get_followers_ids(screen_name='twitter')
+
+    @unittest.skip('skipping non-updated test')
+    def test_lookup_friendships(self):
+        """Test returning relationships of the authenticating user to the
+        comma-separated list of up to 100 screen_names or user_ids provided
+        succeeds"""
+        self.api.lookup_friendships(screen_name='twitter,ryanmcgrath')
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_incoming_friendship_ids(self):
+        """Test returning incoming friendship ids succeeds"""
+        self.api.get_incoming_friendship_ids()
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_outgoing_friendship_ids(self):
+        """Test returning outgoing friendship ids succeeds"""
+        self.api.get_outgoing_friendship_ids()
+
+    @unittest.skip('skipping non-updated test')
+    def test_create_friendship(self):
+        """Test creating a friendship succeeds"""
+        self.api.create_friendship(screen_name='justinbieber')
+
+    @unittest.skip('skipping non-updated test')
+    def test_destroy_friendship(self):
+        """Test destroying a friendship succeeds"""
+        self.api.destroy_friendship(screen_name='justinbieber')
+
+    @unittest.skip('skipping non-updated test')
+    def test_update_friendship(self):
+        """Test updating friendships succeeds"""
+        self.api.update_friendship(screen_name=protected_twitter_1,
+                                   retweets='true')
+
+        self.api.update_friendship(screen_name=protected_twitter_1,
+                                   retweets=False)
+
+    @unittest.skip('skipping non-updated test')
+    def test_show_friendships(self):
+        """Test showing specific friendship succeeds"""
+        self.api.show_friendship(target_screen_name=protected_twitter_1)
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_friends_list(self):
+        """Test getting list of users authenticated user then random user is
+        following succeeds"""
+        self.api.get_friends_list()
+        self.api.get_friends_list(screen_name='twitter')
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_followers_list(self):
+        """Test getting list of users authenticated user then random user are
+        followed by succeeds"""
+        self.api.get_followers_list()
+        self.api.get_followers_list(screen_name='twitter')
+
+    # Users
+    @unittest.skip('skipping non-updated test')
+    def test_get_account_settings(self):
+        """Test getting the authenticated user account settings succeeds"""
+        self.api.get_account_settings()
+
+    @unittest.skip('skipping non-updated test')
+    def test_verify_credentials(self):
+        """Test representation of the authenticated user call succeeds"""
+        self.api.verify_credentials()
+
+    @unittest.skip('skipping non-updated test')
+    def test_update_account_settings(self):
+        """Test updating a user account settings succeeds"""
+        self.api.update_account_settings(lang='en')
+
+    @unittest.skip('skipping non-updated test')
+    def test_update_delivery_service(self):
+        """Test updating delivery settings fails because we don't have
+        a mobile number on the account"""
+        self.assertRaises(TwythonError, self.api.update_delivery_service,
+                          device='none')
+
+    @unittest.skip('skipping non-updated test')
+    def test_update_profile(self):
+        """Test updating profile succeeds"""
+        self.api.update_profile(include_entities='true')
+
+    @unittest.skip('skipping non-updated test')
+    def test_update_profile_colors(self):
+        """Test updating profile colors succeeds"""
+        self.api.update_profile_colors(profile_background_color='3D3D3D')
+
+    @unittest.skip('skipping non-updated test')
+    def test_list_blocks(self):
+        """Test listing users who are blocked by the authenticated user
+        succeeds"""
+        self.api.list_blocks()
+
+    @unittest.skip('skipping non-updated test')
+    def test_list_block_ids(self):
+        """Test listing user ids who are blocked by the authenticated user
+        succeeds"""
+        self.api.list_block_ids()
+
+    @unittest.skip('skipping non-updated test')
+    def test_create_block(self):
+        """Test blocking a user succeeds"""
+        self.api.create_block(screen_name='justinbieber')
+
+    @unittest.skip('skipping non-updated test')
+    def test_destroy_block(self):
+        """Test unblocking a user succeeds"""
+        self.api.destroy_block(screen_name='justinbieber')
+
+    @unittest.skip('skipping non-updated test')
+    def test_lookup_user(self):
+        """Test listing a number of user objects succeeds"""
+        self.api.lookup_user(screen_name='twitter,justinbieber')
+
+    @unittest.skip('skipping non-updated test')
+    def test_show_user(self):
+        """Test showing one user works"""
+        self.api.show_user(screen_name='twitter')
+
+    @unittest.skip('skipping non-updated test')
+    def test_search_users(self):
+        """Test that searching for users succeeds"""
+        self.api.search_users(q='Twitter API')
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_contributees(self):
+        """Test returning list of accounts the specified user can
+        contribute to succeeds"""
+        self.api.get_contributees(screen_name='TechCrunch')
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_contributors(self):
+        """Test returning list of accounts that contribute to the
+        authenticated user fails because we are not a Contributor account"""
+        self.assertRaises(TwythonError, self.api.get_contributors,
+                          screen_name=screen_name)
+
+    @unittest.skip('skipping non-updated test')
+    def test_remove_profile_banner(self):
+        """Test removing profile banner succeeds"""
+        self.api.remove_profile_banner()
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_profile_banner_sizes(self):
+        """Test getting list of profile banner sizes fails because
+        we have not uploaded a profile banner"""
+        self.assertRaises(TwythonError, self.api.get_profile_banner_sizes)
+
+    @unittest.skip('skipping non-updated test')
+    def test_list_mutes(self):
+        """Test listing users who are muted by the authenticated user
+        succeeds"""
+        self.api.list_mutes()
+
+    @unittest.skip('skipping non-updated test')
+    def test_list_mute_ids(self):
+        """Test listing user ids who are muted by the authenticated user
+        succeeds"""
+        self.api.list_mute_ids()
+
+    @unittest.skip('skipping non-updated test')
+    def test_create_mute(self):
+        """Test muting a user succeeds"""
+        self.api.create_mute(screen_name='justinbieber')
+
+    @unittest.skip('skipping non-updated test')
+    def test_destroy_mute(self):
+        """Test muting a user succeeds"""
+        self.api.destroy_mute(screen_name='justinbieber')
+
+    # Suggested Users
+    @unittest.skip('skipping non-updated test')
+    def test_get_user_suggestions_by_slug(self):
+        """Test getting user suggestions by slug succeeds"""
+        self.api.get_user_suggestions_by_slug(slug='twitter')
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_user_suggestions(self):
+        """Test getting user suggestions succeeds"""
+        self.api.get_user_suggestions()
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_user_suggestions_statuses_by_slug(self):
+        """Test getting status of suggested users succeeds"""
+        self.api.get_user_suggestions_statuses_by_slug(slug='funny')
+
+    # Favorites
+    @unittest.skip('skipping non-updated test')
+    def test_get_favorites(self):
+        """Test getting list of favorites for the authenticated
+        user succeeds"""
+        self.api.get_favorites()
+
+    @unittest.skip('skipping non-updated test')
+    def test_create_and_destroy_favorite(self):
+        """Test creating and destroying a favorite on a tweet succeeds"""
+        self.api.create_favorite(id=test_tweet_id)
+        self.api.destroy_favorite(id=test_tweet_id)
+
+    # Lists
+    @unittest.skip('skipping non-updated test')
+    def test_show_lists(self):
+        """Test show lists for specified user"""
+        self.api.show_lists(screen_name='twitter')
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_list_statuses(self):
+        """Test timeline of tweets authored by members of the
+        specified list succeeds"""
+        self.api.get_list_statuses(slug=test_list_slug,
+                                   owner_screen_name=test_list_owner_screen_name)
+
+    @unittest.skip('skipping non-updated test')
+    def test_create_update_destroy_list_add_remove_list_members(self):
+        """Test create a list, adding and removing members then
+        deleting the list succeeds"""
+        the_list = self.api.create_list(name='Stuff %s' % int(time.time()))
+        list_id = the_list['id_str']
+
+        self.api.update_list(list_id=list_id, name='Stuff Renamed \
+                             %s' % int(time.time()))
+
+        screen_names = ['johncena', 'xbox']
+        # Multi add/delete members
+        self.api.create_list_members(list_id=list_id,
+                                     screen_name=screen_names)
+        self.api.delete_list_members(list_id=list_id,
+                                     screen_name=screen_names)
+
+        # Single add/delete member
+        self.api.add_list_member(list_id=list_id, screen_name='justinbieber')
+        self.api.delete_list_member(list_id=list_id,
+                                    screen_name='justinbieber')
+
+        self.api.delete_list(list_id=list_id)
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_list_memberships(self):
+        """Test list of memberhips the authenticated user succeeds"""
+        self.api.get_list_memberships()
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_list_subscribers(self):
+        """Test list of subscribers of a specific list succeeds"""
+        self.api.get_list_subscribers(slug=test_list_slug,
+                                      owner_screen_name=test_list_owner_screen_name)
+
+    @unittest.skip('skipping non-updated test')
+    def test_subscribe_is_subbed_and_unsubscribe_to_list(self):
+        """Test subscribing, is a list sub and unsubbing to list succeeds"""
+        self.api.subscribe_to_list(slug=test_list_slug,
+                                   owner_screen_name=test_list_owner_screen_name)
+        # Returns 404 if user is not a subscriber
+        self.api.is_list_subscriber(slug=test_list_slug,
+                                    owner_screen_name=test_list_owner_screen_name,
+                                    screen_name=screen_name)
+        self.api.unsubscribe_from_list(slug=test_list_slug,
+                                       owner_screen_name=test_list_owner_screen_name)
+
+    @unittest.skip('skipping non-updated test')
+    def test_is_list_member(self):
+        """Test returning if specified user is member of a list succeeds"""
+        # Returns 404 if not list member
+        self.api.is_list_member(slug=test_list_slug,
+                                owner_screen_name=test_list_owner_screen_name,
+                                screen_name='themattharris')
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_list_members(self):
+        """Test listing members of the specified list succeeds"""
+        self.api.get_list_members(slug=test_list_slug,
+                                  owner_screen_name=test_list_owner_screen_name)
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_specific_list(self):
+        """Test getting specific list succeeds"""
+        self.api.get_specific_list(slug=test_list_slug,
+                                   owner_screen_name=test_list_owner_screen_name)
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_list_subscriptions(self):
+        """Test collection of the lists the specified user is
+        subscribed to succeeds"""
+        self.api.get_list_subscriptions(screen_name='twitter')
+
+    @unittest.skip('skipping non-updated test')
+    def test_show_owned_lists(self):
+        """Test collection of lists the specified user owns succeeds"""
+        self.api.show_owned_lists(screen_name='twitter')
+
+    # Saved Searches
+    @unittest.skip('skipping non-updated test')
+    def test_get_saved_searches(self):
+        """Test getting list of saved searches for authenticated
+        user succeeds"""
+        self.api.get_saved_searches()
+
+    @unittest.skip('skipping non-updated test')
+    def test_create_get_destroy_saved_search(self):
+        """Test getting list of saved searches for authenticated
+        user succeeds"""
+        saved_search = self.api.create_saved_search(query='#Twitter')
+        saved_search_id = saved_search['id_str']
+
+        self.api.show_saved_search(id=saved_search_id)
+        self.api.destroy_saved_search(id=saved_search_id)
+
+    # Places & Geo
+    @unittest.skip('skipping non-updated test')
+    def test_get_geo_info(self):
+        """Test getting info about a geo location succeeds"""
+        self.api.get_geo_info(place_id='df51dec6f4ee2b2c')
+
+    @unittest.skip('skipping non-updated test')
+    def test_reverse_geo_code(self):
+        """Test reversing geocode succeeds"""
+        self.api.reverse_geocode(lat='37.76893497', long='-122.42284884')
+
+    @unittest.skip('skipping non-updated test')
+    def test_search_geo(self):
+        """Test search for places that can be attached
+        to a statuses/update succeeds"""
+        self.api.search_geo(query='Toronto')
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_similar_places(self):
+        """Test locates places near the given coordinates which
+        are similar in name succeeds"""
+        self.api.get_similar_places(lat='37', long='-122', name='Twitter HQ')
+
+    # Trends
+    @unittest.skip('skipping non-updated test')
+    def test_get_place_trends(self):
+        """Test getting the top 10 trending topics for a specific
+        WOEID succeeds"""
+        self.api.get_place_trends(id=1)
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_available_trends(self):
+        """Test returning locations that Twitter has trending
+        topic information for succeeds"""
+        self.api.get_available_trends()
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_closest_trends(self):
+        """Test getting the locations that Twitter has trending topic
+        information for, closest to a specified location succeeds"""
+        self.api.get_closest_trends(lat='37', long='-122')
+
+    # Help
+    @unittest.skip('skipping non-updated test')
+    def test_get_twitter_configuration(self):
+        """Test getting Twitter's configuration succeeds"""
+        self.api.get_twitter_configuration()
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_supported_languages(self):
+        """Test getting languages supported by Twitter succeeds"""
+        self.api.get_supported_languages()
+
+    @unittest.skip('skipping non-updated test')
+    def test_privacy_policy(self):
+        """Test getting Twitter's Privacy Policy succeeds"""
+        self.api.get_privacy_policy()
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_tos(self):
+        """Test getting the Twitter Terms of Service succeeds"""
+        self.api.get_tos()
+
+    @unittest.skip('skipping non-updated test')
+    def test_get_application_rate_limit_status(self):
+        """Test getting application rate limit status succeeds"""
+        self.oauth2_api.get_application_rate_limit_status()

tests/test_html_for_tweet.py

@@ -0,0 +1,130 @@
+# -*- coding: utf-8 -*-
+import json
+import os
+
+from twython import Twython, TwythonError
+
+from .config import unittest
+
+
+class TestHtmlForTweetTestCase(unittest.TestCase):
+    def setUp(self):
+        self.api = Twython('', '', '', '')
+
+    def load_tweet(self, name):
+        f = open(os.path.join(
+                os.path.dirname(__file__),
+                'tweets',
+                '%s.json' % name
+            ))
+        tweet = json.load(f)
+        f.close()
+        return tweet
+
+    def test_basic(self):
+        """Test HTML for Tweet returns what we want"""
+        tweet_object = self.load_tweet('basic')
+        tweet_text = self.api.html_for_tweet(tweet_object)
+        self.assertEqual(tweet_text,
+            '<a href="http://t.co/FCmXyI6VHd" class="twython-url">google.com</a> is a <a href="https://twitter.com/search?q=%23cool" class="twython-hashtag">#cool</a> site, lol! <a href="https://twitter.com/mikehelmick" class="twython-mention">@mikehelmick</a> shd <a href="https://twitter.com/search?q=%23checkitout" class="twython-hashtag">#checkitout</a>. Love, <a href="https://twitter.com/__twython__" class="twython-mention">@__twython__</a> <a href="https://t.co/67pwRvY6z9" class="twython-url">github.com</a> <a href="http://t.co/N6InAO4B71" class="twython-media">pic.twitter.com/N6InAO4B71</a>')
+
+    def test_reply(self):
+        """Test HTML for Tweet links the replied-to username."""
+        tweet_object = self.load_tweet('reply')
+        tweet_text = self.api.html_for_tweet(tweet_object)
+        self.assertEqual(tweet_text,
+                u'<span class="twython-tweet-prefix"><a href="https://twitter.com/philgyford" class="twython-mention">@philgyford</a> </span>Here’s a test tweet that goes on as much as possible and includes an image. Hi to my fans in testland!<span class="twython-tweet-suffix"> https://t.co/tzhyk2QWSr</span>')
+
+    def test_expanded_url(self):
+        """Test using expanded url in HTML for Tweet displays full urls"""
+        tweet_object = self.load_tweet('basic')
+        tweet_text = self.api.html_for_tweet(tweet_object,
+                                             use_expanded_url=True)
+        # Make sure full url is in HTML
+        self.assertTrue('http://google.com' in tweet_text)
+
+    def test_short_url(self):
+        """Test using expanded url in HTML for Tweet displays full urls"""
+        tweet_object = self.load_tweet('basic')
+        tweet_text = self.api.html_for_tweet(tweet_object, False)
+        # Make sure HTML doesn't contain the display OR expanded url
+        self.assertTrue('http://google.com' not in tweet_text)
+        self.assertTrue('google.com' not in tweet_text)
+
+    def test_identical_urls(self):
+        """If the 'url's for different url entities are identical, they should link correctly."""
+        tweet_object = self.load_tweet('identical_urls')
+        tweet_text = self.api.html_for_tweet(tweet_object)
+        self.assertEqual(tweet_text,
+                u'Use Cases, Trials and Making 5G a Reality <a href="https://t.co/W0uArTMk9N" class="twython-url">buff.ly/2sEhrgO</a> #5G #innovation via @5GWorldSeries <a href="https://t.co/W0uArTMk9N" class="twython-url">buff.ly/2sEhrgO</a>')
+
+    def test_symbols(self):
+        tweet_object = self.load_tweet('symbols')
+        tweet_text = self.api.html_for_tweet(tweet_object)
+        # Should only link symbols listed in entities:
+        self.assertTrue('<a href="https://twitter.com/search?q=%24AAPL" class="twython-symbol">$AAPL</a>' in tweet_text)
+        self.assertTrue('<a href="https://twitter.com/search?q=%24ANOTHER" class="twython-symbol">$ANOTHER</a>' not in tweet_text)
+
+    def test_no_symbols(self):
+        """Should still work if tweet object has no symbols list"""
+        tweet = self.load_tweet('symbols')
+        # Save a copy:
+        symbols = tweet['entities']['symbols']
+        del tweet['entities']['symbols']
+        tweet_text = self.api.html_for_tweet(tweet)
+        self.assertTrue('symbols: $AAPL and' in tweet_text)
+        self.assertTrue('and $ANOTHER and $A.' in tweet_text)
+
+    def test_compatmode(self):
+        tweet_object = self.load_tweet('compat')
+        tweet_text = self.api.html_for_tweet(tweet_object)
+        # link to compat web status link
+        self.assertTrue(
+            u'<a href="https://t.co/SRmsuks2ru" class="twython-url">twitter.com/i/web/status/7…</a>' in tweet_text)
+
+    def test_extendedmode(self):
+        tweet_object = self.load_tweet('extended')
+        tweet_text = self.api.html_for_tweet(tweet_object)
+        # full tweet rendered with suffix
+        self.assertEqual(tweet_text,
+            'Say more about what\'s happening! Rolling out now: photos, videos, GIFs, polls, and Quote Tweets no longer count toward your 140 characters.<span class="twython-tweet-suffix"> <a href="https://t.co/I9pUC0NdZC" class="twython-media">pic.twitter.com/I9pUC0NdZC</a></span>')
+
+    def test_entities_with_prefix(self):
+        """
+        If there is a username mention at the start of a tweet it's in the
+        "prefix" and so isn't part of the main tweet display text.
+        But its length is still counted in the indices of any subsequent
+        mentions, urls, hashtags, etc.
+        """
+        self.maxDiff = 2200
+        tweet_object = self.load_tweet('entities_with_prefix')
+        tweet_text = self.api.html_for_tweet(tweet_object)
+        self.assertEqual(tweet_text,
+            u'<span class="twython-tweet-prefix"><a href="https://twitter.com/philgyford" class="twython-mention">@philgyford</a> </span>This is a test for <a href="https://twitter.com/visionphil" class="twython-mention">@visionphil</a> that includes a link <a href="https://t.co/sKw4J3A8SZ" class="twython-url">example.org</a> and <a href="https://twitter.com/search?q=%23hashtag" class="twython-hashtag">#hashtag</a> and X for good measure AND that is longer than 140 characters. <a href="https://t.co/jnQdy7Zg7u" class="twython-url">example.com</a>')
+
+    def test_media(self):
+        tweet_object = self.load_tweet('media')
+        tweet_text = self.api.html_for_tweet(tweet_object)
+
+        self.assertEqual(
+            u"""I made some D3.js charts showing the years covered by books in a series compared to their publishing dates <a href="https://t.co/2yUmmn3TOc" class="twython-url">gyford.com/phil/writing/2\u2026</a><span class="twython-tweet-suffix"> <a href="https://t.co/OwNc6uJklg" class="twython-media">pic.twitter.com/OwNc6uJklg</a></span>""",
+            tweet_text)
+
+    def test_quoted(self):
+        "With expand_quoted_status=True it should include a quoted tweet."
+        tweet_object = self.load_tweet('quoted')
+        tweet_text = self.api.html_for_tweet(tweet_object,
+                                             expand_quoted_status=True)
+        self.assertEqual(
+                u"""Here\u2019s a quoted tweet. <a href="https://t.co/3neKzof0gT" class="twython-url">twitter.com/philgyford/sta\u2026</a><blockquote class="twython-quote">The quoted tweet text.<cite><a href="https://twitter.com/philgyford/status/917699069916729344"><span class="twython-quote-user-name">Phil Gyford</span><span class="twython-quote-user-screenname">@philgyford</span></a></cite></blockquote>""",
+                tweet_text)
+
+    def test_retweet(self):
+        "With expand_quoted_status=True it should include a quoted tweet."
+        tweet_object = self.load_tweet('retweet')
+        tweet_text = self.api.html_for_tweet(tweet_object)
+
+        self.assertEqual(
+                u"""My aunt and uncle in a very ill humour one with another, but I made shift with much ado to keep them from scolding.""",
+                tweet_text)
+

tests/test_streaming.py

@@ -1,11 +1,9 @@
 from twython import TwythonStreamer, TwythonStreamError
 
 from .config import (
-    app_key, app_secret, oauth_token, oauth_token_secret
+    app_key, app_secret, oauth_token, oauth_token_secret, unittest
 )
 
-import unittest
-
 
 class TwythonStreamTestCase(unittest.TestCase):
     def setUp(self):
@@ -29,19 +27,24 @@ class TwythonStreamTestCase(unittest.TestCase):
                                           oauth_token, oauth_token_secret,
                                           client_args=client_args)
 
+    @unittest.skip('skipping non-updated test')
     def test_stream_status_filter(self):
         self.api.statuses.filter(track='twitter')
 
+    @unittest.skip('skipping non-updated test')
     def test_stream_status_sample(self):
         self.api.statuses.sample()
 
+    @unittest.skip('skipping non-updated test')
     def test_stream_status_firehose(self):
         self.assertRaises(TwythonStreamError, self.api.statuses.firehose,
                           track='twitter')
 
+    @unittest.skip('skipping non-updated test')
     def test_stream_site(self):
         self.assertRaises(TwythonStreamError, self.api.site,
                           follow='twitter')
 
+    @unittest.skip('skipping non-updated test')
     def test_stream_user(self):
         self.api.user(track='twitter')

tests/tweets/basic.json

@@ -0,0 +1,170 @@
+{
+  "contributors":null,
+  "truncated":false,
+  "text":"http://t.co/FCmXyI6VHd is a #cool site, lol! @mikehelmick shd #checkitout. Love, @__twython__ https://t.co/67pwRvY6z9 http://t.co/N6InAO4B71",
+  "in_reply_to_status_id":null,
+  "id":349683012054683648,
+  "favorite_count":0,
+  "source":"<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>",
+  "retweeted":false,
+  "coordinates":null,
+  "entities":{
+    "symbols":[
+
+    ],
+    "user_mentions":[
+      {
+        "id":29251354,
+        "indices":[
+          45,
+          57
+        ],
+        "id_str":"29251354",
+        "screen_name":"mikehelmick",
+        "name":"Mike Helmick"
+      },
+      {
+        "id":1431865928,
+        "indices":[
+          81,
+          93
+        ],
+        "id_str":"1431865928",
+        "screen_name":"__twython__",
+        "name":"Twython"
+      }
+    ],
+    "hashtags":[
+      {
+        "indices":[
+          28,
+          33
+        ],
+        "text":"cool"
+      },
+      {
+        "indices":[
+          62,
+          73
+        ],
+        "text":"checkitout"
+      }
+    ],
+    "urls":[
+      {
+        "url":"http://t.co/FCmXyI6VHd",
+        "indices":[
+          0,
+          22
+        ],
+        "expanded_url":"http://google.com",
+        "display_url":"google.com"
+      },
+      {
+        "url":"https://t.co/67pwRvY6z9",
+        "indices":[
+          94,
+          117
+        ],
+        "expanded_url":"https://github.com",
+        "display_url":"github.com"
+      }
+    ],
+    "media":[
+      {
+        "id":537884378513162240,
+        "id_str":"537884378513162240",
+        "indices":[
+          118,
+          140
+        ],
+        "media_url":"http://pbs.twimg.com/media/B3by_g-CQAAhrO5.jpg",
+        "media_url_https":"https://pbs.twimg.com/media/B3by_g-CQAAhrO5.jpg",
+        "url":"http://t.co/N6InAO4B71",
+        "display_url":"pic.twitter.com/N6InAO4B71",
+        "expanded_url":"http://twitter.com/pingofglitch/status/537884380060844032/photo/1",
+        "type":"photo",
+        "sizes":{
+          "large":{
+            "w":1024,
+            "h":640,
+            "resize":"fit"
+          },
+          "thumb":{
+            "w":150,
+            "h":150,
+            "resize":"crop"
+          },
+          "medium":{
+            "w":600,
+            "h":375,
+            "resize":"fit"
+          },
+          "small":{
+            "w":340,
+            "h":212,
+            "resize":"fit"
+          }
+        }
+      }
+    ]
+  },
+  "in_reply_to_screen_name":null,
+  "id_str":"349683012054683648",
+  "retweet_count":0,
+  "in_reply_to_user_id":null,
+  "favorited":false,
+  "user":{
+    "follow_request_sent":false,
+    "profile_use_background_image":true,
+    "default_profile_image":true,
+    "id":1431865928,
+    "verified":false,
+    "profile_text_color":"333333",
+    "profile_image_url_https":"https://si0.twimg.com/sticky/default_profile_images/default_profile_3_normal.png",
+    "profile_sidebar_fill_color":"DDEEF6",
+    "entities":{
+      "description":{
+        "urls":[
+
+        ]
+      }
+    },
+    "followers_count":1,
+    "profile_sidebar_border_color":"C0DEED",
+    "id_str":"1431865928",
+    "profile_background_color":"3D3D3D",
+    "listed_count":0,
+    "profile_background_image_url_https":"https://si0.twimg.com/images/themes/theme1/bg.png",
+    "utc_offset":null,
+    "statuses_count":2,
+    "description":"",
+    "friends_count":1,
+    "location":"",
+    "profile_link_color":"0084B4",
+    "profile_image_url":"http://a0.twimg.com/sticky/default_profile_images/default_profile_3_normal.png",
+    "following":false,
+    "geo_enabled":false,
+    "profile_background_image_url":"http://a0.twimg.com/images/themes/theme1/bg.png",
+    "screen_name":"__twython__",
+    "lang":"en",
+    "profile_background_tile":false,
+    "favourites_count":0,
+    "name":"Twython",
+    "notifications":false,
+    "url":null,
+    "created_at":"Thu May 16 01:11:09 +0000 2013",
+    "contributors_enabled":false,
+    "time_zone":null,
+    "protected":false,
+    "default_profile":false,
+    "is_translator":false
+  },
+  "geo":null,
+  "in_reply_to_user_id_str":null,
+  "possibly_sensitive":false,
+  "lang":"en",
+  "created_at":"Wed Jun 26 00:18:21 +0000 2013",
+  "in_reply_to_status_id_str":null,
+  "place":null
+}

tests/tweets/compat.json

@@ -0,0 +1,51 @@
+{
+  "contributors":null,
+  "truncated":true,
+  "text":"Say more about what's happening! Rolling out now: photos, videos, GIFs, polls, and Quote Tweets no longer count tow\u2026 https://t.co/SRmsuks2ru",
+  "is_quote_status":false,
+  "in_reply_to_status_id":null,
+  "id":777915304261193728,
+  "favorite_count":13856,
+  "source":"<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>",
+  "retweeted":false,
+  "coordinates":null,
+  "entities":{
+    "symbols":[
+
+    ],
+    "user_mentions":[
+
+    ],
+    "hashtags":[
+
+    ],
+    "urls":[
+      {
+        "url":"https://t.co/SRmsuks2ru",
+        "indices":[
+          117,
+          140
+        ],
+        "expanded_url":"https://twitter.com/i/web/status/777915304261193728",
+        "display_url":"twitter.com/i/web/status/7\u2026"
+      }
+    ]
+  },
+  "in_reply_to_screen_name":null,
+  "id_str":"777915304261193728",
+  "retweet_count":14767,
+  "in_reply_to_user_id":null,
+  "favorited":false,
+  "user":{
+    "name":"Twitter",
+    "screen_name":"twitter"
+  },
+  "geo":null,
+  "in_reply_to_user_id_str":null,
+  "possibly_sensitive":false,
+  "possibly_sensitive_appealable":false,
+  "lang":"en",
+  "created_at":"Mon Sep 19 17:00:36 +0000 2016",
+  "in_reply_to_status_id_str":null,
+  "place":null
+}

tests/tweets/entities_with_prefix.json

@@ -0,0 +1,68 @@
+{  
+  "created_at":"Sat Jan 06 18:56:35 +0000 2018",
+  "id":949716340755091458,
+  "id_str":"949716340755091458",
+  "full_text":"@philgyford This is a test for @visionphil that includes a link https://t.co/sKw4J3A8SZ and #hashtag and X for good measure AND that is longer than 140 characters. https://t.co/jnQdy7Zg7u",
+  "truncated":false,
+  "display_text_range":[ 12, 187 ],
+  "entities":{  
+    "hashtags":[  
+      {  
+        "text":"hashtag",
+        "indices":[ 92, 100 ]
+      }
+    ],
+    "symbols":[  ],
+    "user_mentions":[  
+      {  
+        "screen_name":"philgyford",
+        "name":"Phil Gyford",
+        "id":12552,
+        "id_str":"12552",
+        "indices":[ 0, 11 ]
+      },
+      {  
+        "screen_name":"visionphil",
+        "name":"Vision Phil",
+        "id":104456050,
+        "id_str":"104456050",
+        "indices":[ 31, 42 ]
+      }
+    ],
+    "urls":[  
+      {  
+        "url":"https://t.co/sKw4J3A8SZ",
+        "expanded_url":"http://example.org",
+        "display_url":"example.org",
+        "indices":[ 64, 87 ]
+      },
+      {  
+        "url":"https://t.co/jnQdy7Zg7u",
+        "expanded_url":"http://example.com",
+        "display_url":"example.com",
+        "indices":[ 164, 187 ]
+      }
+    ]
+  },
+  "source":"<a href=\"http://tapbots.com/software/tweetbot/mac\" rel=\"nofollow\">Tweetbot for Mac</a>",
+  "in_reply_to_status_id":948561036889722880,
+  "in_reply_to_status_id_str":"948561036889722880",
+  "in_reply_to_user_id":12552,
+  "in_reply_to_user_id_str":"12552",
+  "in_reply_to_screen_name":"philgyford",
+  "user":{  
+    "id":2030131,
+    "id_str":"2030131"
+  },
+  "geo":null,
+  "coordinates":null,
+  "place":null,
+  "contributors":null,
+  "is_quote_status":false,
+  "retweet_count":0,
+  "favorite_count":0,
+  "favorited":false,
+  "retweeted":false,
+  "possibly_sensitive":false,
+  "lang":"en"
+}

tests/tweets/extended.json

@@ -0,0 +1,204 @@
+{
+  "full_text":"Say more about what's happening! Rolling out now: photos, videos, GIFs, polls, and Quote Tweets no longer count toward your 140 characters. https://t.co/I9pUC0NdZC",
+  "truncated":false,
+  "is_quote_status":false,
+  "in_reply_to_status_id":null,
+  "id":777915304261193728,
+  "favorite_count":13856,
+  "contributors":null,
+  "source":"<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>",
+  "retweeted":false,
+  "coordinates":null,
+  "entities":{
+    "symbols":[
+
+    ],
+    "user_mentions":[
+
+    ],
+    "hashtags":[
+
+    ],
+    "urls":[
+
+    ],
+    "media":[
+      {
+        "expanded_url":"https://twitter.com/twitter/status/777915304261193728/photo/1",
+        "sizes":{
+          "small":{
+            "h":340,
+            "w":340,
+            "resize":"fit"
+          },
+          "large":{
+            "h":700,
+            "w":700,
+            "resize":"fit"
+          },
+          "medium":{
+            "h":600,
+            "w":600,
+            "resize":"fit"
+          },
+          "thumb":{
+            "h":150,
+            "w":150,
+            "resize":"crop"
+          }
+        },
+        "url":"https://t.co/I9pUC0NdZC",
+        "media_url_https":"https://pbs.twimg.com/tweet_video_thumb/Csu1TzEVMAAAEv7.jpg",
+        "id_str":"777914712382058496",
+        "indices":[
+          140,
+          163
+        ],
+        "media_url":"http://pbs.twimg.com/tweet_video_thumb/Csu1TzEVMAAAEv7.jpg",
+        "type":"photo",
+        "id":777914712382058496,
+        "display_url":"pic.twitter.com/I9pUC0NdZC"
+      }
+    ]
+  },
+  "in_reply_to_screen_name":null,
+  "id_str":"777915304261193728",
+  "display_text_range":[
+    0,
+    139
+  ],
+  "retweet_count":14767,
+  "in_reply_to_user_id":null,
+  "favorited":false,
+  "user":{
+    "follow_request_sent":false,
+    "has_extended_profile":false,
+    "profile_use_background_image":true,
+    "id":783214,
+    "verified":true,
+    "profile_text_color":"333333",
+    "profile_image_url_https":"https://pbs.twimg.com/profile_images/767879603977191425/29zfZY6I_normal.jpg",
+    "profile_sidebar_fill_color":"F6F6F6",
+    "is_translator":false,
+    "geo_enabled":true,
+    "entities":{
+      "url":{
+        "urls":[
+          {
+            "url":"http://t.co/5iRhy7wTgu",
+            "indices":[
+              0,
+              22
+            ],
+            "expanded_url":"http://blog.twitter.com/",
+            "display_url":"blog.twitter.com"
+          }
+        ]
+      },
+      "description":{
+        "urls":[
+          {
+            "url":"https://t.co/qq1HEzvnrA",
+            "indices":[
+              84,
+              107
+            ],
+            "expanded_url":"http://support.twitter.com",
+            "display_url":"support.twitter.com"
+          }
+        ]
+      }
+    },
+    "followers_count":56827498,
+    "protected":false,
+    "location":"San Francisco, CA",
+    "default_profile_image":false,
+    "id_str":"783214",
+    "lang":"en",
+    "utc_offset":-25200,
+    "statuses_count":3161,
+    "description":"Your official source for news, updates and tips from Twitter, Inc. Need help? Visit https://t.co/qq1HEzvnrA.",
+    "friends_count":145,
+    "profile_link_color":"226699",
+    "profile_image_url":"http://pbs.twimg.com/profile_images/767879603977191425/29zfZY6I_normal.jpg",
+    "notifications":false,
+    "profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/657090062/l1uqey5sy82r9ijhke1i.png",
+    "profile_background_color":"ACDED6",
+    "profile_banner_url":"https://pbs.twimg.com/profile_banners/783214/1471929200",
+    "profile_background_image_url":"http://pbs.twimg.com/profile_background_images/657090062/l1uqey5sy82r9ijhke1i.png",
+    "name":"Twitter",
+    "is_translation_enabled":false,
+    "profile_background_tile":true,
+    "favourites_count":2332,
+    "screen_name":"twitter",
+    "url":"http://t.co/5iRhy7wTgu",
+    "created_at":"Tue Feb 20 14:35:54 +0000 2007",
+    "contributors_enabled":false,
+    "time_zone":"Pacific Time (US & Canada)",
+    "profile_sidebar_border_color":"FFFFFF",
+    "default_profile":false,
+    "following":false,
+    "listed_count":90445
+  },
+  "geo":null,
+  "in_reply_to_user_id_str":null,
+  "possibly_sensitive":false,
+  "possibly_sensitive_appealable":false,
+  "lang":"en",
+  "created_at":"Mon Sep 19 17:00:36 +0000 2016",
+  "in_reply_to_status_id_str":null,
+  "place":null,
+  "extended_entities":{
+    "media":[
+      {
+        "expanded_url":"https://twitter.com/twitter/status/777915304261193728/photo/1",
+        "display_url":"pic.twitter.com/I9pUC0NdZC",
+        "url":"https://t.co/I9pUC0NdZC",
+        "media_url_https":"https://pbs.twimg.com/tweet_video_thumb/Csu1TzEVMAAAEv7.jpg",
+        "video_info":{
+          "aspect_ratio":[
+            1,
+            1
+          ],
+          "variants":[
+            {
+              "url":"https://pbs.twimg.com/tweet_video/Csu1TzEVMAAAEv7.mp4",
+              "bitrate":0,
+              "content_type":"video/mp4"
+            }
+          ]
+        },
+        "id_str":"777914712382058496",
+        "sizes":{
+          "small":{
+            "h":340,
+            "w":340,
+            "resize":"fit"
+          },
+          "large":{
+            "h":700,
+            "w":700,
+            "resize":"fit"
+          },
+          "medium":{
+            "h":600,
+            "w":600,
+            "resize":"fit"
+          },
+          "thumb":{
+            "h":150,
+            "w":150,
+            "resize":"crop"
+          }
+        },
+        "indices":[
+          140,
+          163
+        ],
+        "type":"animated_gif",
+        "id":777914712382058496,
+        "media_url":"http://pbs.twimg.com/tweet_video_thumb/Csu1TzEVMAAAEv7.jpg"
+      }
+    ]
+  }
+}

tests/tweets/identical_urls.json

@@ -0,0 +1,34 @@
+{
+  "entities":{
+    "hashtags":[
+
+    ],
+    "user_mentions":[
+
+    ],
+    "symbols":[
+
+    ],
+    "urls":[
+      {
+        "display_url":"buff.ly/2sEhrgO",
+        "expanded_url":"http://buff.ly/2sEhrgO",
+        "indices":[
+          42,
+          65
+        ],
+        "url":"https://t.co/W0uArTMk9N"
+      },
+      {
+        "display_url":"buff.ly/2sEhrgO",
+        "expanded_url":"http://buff.ly/2sEhrgO",
+        "indices":[
+          101,
+          124
+        ],
+        "url":"https://t.co/W0uArTMk9N"
+      }
+    ]
+  },
+  "full_text":"Use Cases, Trials and Making 5G a Reality https://t.co/W0uArTMk9N #5G #innovation via @5GWorldSeries https://t.co/W0uArTMk9N"
+}

tests/tweets/media.json

@@ -0,0 +1,205 @@
+{
+  "contributors":null,
+  "truncated":false,
+  "is_quote_status":false,
+  "in_reply_to_status_id":null,
+  "id":905105588279013377,
+  "favorite_count":10,
+  "full_text":"I made some D3.js charts showing the years covered by books in a series compared to their publishing dates https://t.co/2yUmmn3TOc https://t.co/OwNc6uJklg",
+  "source":"<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>",
+  "retweeted":false,
+  "coordinates":null,
+  "entities":{
+    "symbols":[
+
+    ],
+    "user_mentions":[
+
+    ],
+    "hashtags":[
+
+    ],
+    "urls":[
+      {
+        "url":"https://t.co/2yUmmn3TOc",
+        "indices":[
+          107,
+          130
+        ],
+        "expanded_url":"http://www.gyford.com/phil/writing/2017/09/05/book-series-charts.php",
+        "display_url":"gyford.com/phil/writing/2\u2026"
+      }
+    ],
+    "media":[
+      {
+        "expanded_url":"https://twitter.com/philgyford/status/905105588279013377/photo/1",
+        "display_url":"pic.twitter.com/OwNc6uJklg",
+        "url":"https://t.co/OwNc6uJklg",
+        "media_url_https":"https://pbs.twimg.com/media/DI-UuNlWAAA_cvz.jpg",
+        "id_str":"905105571765944320",
+        "sizes":{
+          "small":{
+            "h":256,
+            "resize":"fit",
+            "w":680
+          },
+          "large":{
+            "h":376,
+            "resize":"fit",
+            "w":1000
+          },
+          "medium":{
+            "h":376,
+            "resize":"fit",
+            "w":1000
+          },
+          "thumb":{
+            "h":150,
+            "resize":"crop",
+            "w":150
+          }
+        },
+        "indices":[
+          131,
+          154
+        ],
+        "type":"photo",
+        "id":905105571765944320,
+        "media_url":"http://pbs.twimg.com/media/DI-UuNlWAAA_cvz.jpg"
+      }
+    ]
+  },
+  "in_reply_to_screen_name":null,
+  "in_reply_to_user_id":null,
+  "display_text_range":[
+    0,
+    130
+  ],
+  "retweet_count":1,
+  "id_str":"905105588279013377",
+  "favorited":false,
+  "user":{
+    "screen_name":"philgyford",
+    "name":"Phil Gyford"
+  },
+  "geo":null,
+  "in_reply_to_user_id_str":null,
+  "possibly_sensitive":false,
+  "possibly_sensitive_appealable":false,
+  "lang":"en",
+  "created_at":"Tue Sep 05 16:29:22 +0000 2017",
+  "in_reply_to_status_id_str":null,
+  "place":null,
+  "extended_entities":{
+    "media":[
+      {
+        "expanded_url":"https://twitter.com/philgyford/status/905105588279013377/photo/1",
+        "display_url":"pic.twitter.com/OwNc6uJklg",
+        "url":"https://t.co/OwNc6uJklg",
+        "media_url_https":"https://pbs.twimg.com/media/DI-UuNlWAAA_cvz.jpg",
+        "id_str":"905105571765944320",
+        "sizes":{
+          "small":{
+            "h":256,
+            "resize":"fit",
+            "w":680
+          },
+          "large":{
+            "h":376,
+            "resize":"fit",
+            "w":1000
+          },
+          "medium":{
+            "h":376,
+            "resize":"fit",
+            "w":1000
+          },
+          "thumb":{
+            "h":150,
+            "resize":"crop",
+            "w":150
+          }
+        },
+        "indices":[
+          131,
+          154
+        ],
+        "type":"photo",
+        "id":905105571765944320,
+        "media_url":"http://pbs.twimg.com/media/DI-UuNlWAAA_cvz.jpg"
+      },
+      {
+        "expanded_url":"https://twitter.com/philgyford/status/905105588279013377/photo/1",
+        "display_url":"pic.twitter.com/OwNc6uJklg",
+        "url":"https://t.co/OwNc6uJklg",
+        "media_url_https":"https://pbs.twimg.com/media/DI-UuQbXUAQ1WlR.jpg",
+        "id_str":"905105572529393668",
+        "sizes":{
+          "large":{
+            "h":399,
+            "resize":"fit",
+            "w":1000
+          },
+          "small":{
+            "h":271,
+            "resize":"fit",
+            "w":680
+          },
+          "medium":{
+            "h":399,
+            "resize":"fit",
+            "w":1000
+          },
+          "thumb":{
+            "h":150,
+            "resize":"crop",
+            "w":150
+          }
+        },
+        "indices":[
+          131,
+          154
+        ],
+        "type":"photo",
+        "id":905105572529393668,
+        "media_url":"http://pbs.twimg.com/media/DI-UuQbXUAQ1WlR.jpg"
+      },
+      {
+        "expanded_url":"https://twitter.com/philgyford/status/905105588279013377/photo/1",
+        "display_url":"pic.twitter.com/OwNc6uJklg",
+        "url":"https://t.co/OwNc6uJklg",
+        "media_url_https":"https://pbs.twimg.com/media/DI-UuTIXcAA-t1_.jpg",
+        "id_str":"905105573255016448",
+        "sizes":{
+          "small":{
+            "h":195,
+            "resize":"fit",
+            "w":680
+          },
+          "large":{
+            "h":287,
+            "resize":"fit",
+            "w":1000
+          },
+          "medium":{
+            "h":287,
+            "resize":"fit",
+            "w":1000
+          },
+          "thumb":{
+            "h":150,
+            "resize":"crop",
+            "w":150
+          }
+        },
+        "indices":[
+          131,
+          154
+        ],
+        "type":"photo",
+        "id":905105573255016448,
+        "media_url":"http://pbs.twimg.com/media/DI-UuTIXcAA-t1_.jpg"
+      }
+    ]
+  }
+}

tests/tweets/quoted.json

@@ -0,0 +1,94 @@
+{  
+  "contributors":null,
+  "truncated":false,
+  "text":"Here\u2019s a quoted tweet. https://t.co/3neKzof0gT",
+  "is_quote_status":true,
+  "in_reply_to_status_id":null,
+  "id":917706313785905157,
+  "favorite_count":0,
+  "source":"<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>",
+  "quoted_status_id":917699069916729344,
+  "retweeted":false,
+  "coordinates":null,
+  "quoted_status":{  
+    "contributors":null,
+    "truncated":false,
+    "text":"The quoted tweet text.",
+    "is_quote_status":false,
+    "in_reply_to_status_id":null,
+    "id":917699069916729344,
+    "favorite_count":1,
+    "source":"<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>",
+    "retweeted":false,
+    "coordinates":null,
+    "entities":{  
+      "symbols":[  
+
+      ],
+      "user_mentions":[  
+
+      ],
+      "hashtags":[  
+
+      ],
+      "urls":[  
+
+      ]
+    },
+    "in_reply_to_screen_name":null,
+    "in_reply_to_user_id":null,
+    "retweet_count":0,
+    "id_str":"917699069916729344",
+    "favorited":false,
+    "user":{  
+      "screen_name":"philgyford",
+      "name":"Phil Gyford"
+    },
+    "geo":null,
+    "in_reply_to_user_id_str":null,
+    "lang":"ht",
+    "created_at":"Tue Oct 10 10:31:22 +0000 2017",
+    "in_reply_to_status_id_str":null,
+    "place":null
+  },
+  "entities":{  
+    "symbols":[  
+
+    ],
+    "user_mentions":[  
+
+    ],
+    "hashtags":[  
+
+    ],
+    "urls":[  
+      {  
+        "url":"https://t.co/3neKzof0gT",
+        "indices":[  
+          23,
+          46
+        ],
+        "expanded_url":"https://twitter.com/philgyford/status/917699069916729344",
+        "display_url":"twitter.com/philgyford/sta\u2026"
+      }
+    ]
+  },
+  "in_reply_to_screen_name":null,
+  "in_reply_to_user_id":null,
+  "retweet_count":0,
+  "id_str":"917706313785905157",
+  "favorited":false,
+  "user":{  
+    "screen_name":"philgyfordtest",
+    "name":"Phil Gyford Test"
+  },
+  "geo":null,
+  "in_reply_to_user_id_str":null,
+  "possibly_sensitive":false,
+  "possibly_sensitive_appealable":false,
+  "lang":"en",
+  "created_at":"Tue Oct 10 11:00:10 +0000 2017",
+  "quoted_status_id_str":"917699069916729344",
+  "in_reply_to_status_id_str":null,
+  "place":null
+}

tests/tweets/reply.json

@@ -0,0 +1,62 @@
+{
+  "display_text_range":[
+    12,
+    114
+  ],
+  "in_reply_to_status_id_str":"742374355531923456",
+  "source":"<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>",
+  "geo":null,
+  "full_text":"@philgyford Here\u2019s a test tweet that goes on as much as possible and includes an image. Hi to my fans in testland! https://t.co/tzhyk2QWSr",
+  "extended_entities":{
+    "media":[
+
+    ]
+  },
+  "id_str":"300",
+  "in_reply_to_status_id":742374355531923456,
+  "id":300,
+  "in_reply_to_screen_name":"philgyford",
+  "retweet_count":0,
+  "user":{
+
+  },
+  "created_at":"Mon Jun 13 15:48:06 +0000 2016",
+  "lang":"en",
+  "favorite_count":0,
+  "coordinates":null,
+  "place":null,
+  "contributors":null,
+  "in_reply_to_user_id":12552,
+  "in_reply_to_user_id_str":"12552",
+  "retweeted":false,
+  "favorited":false,
+  "truncated":false,
+  "entities":{
+    "user_mentions":[
+      {
+        "id_str":"12552",
+        "id":12552,
+        "screen_name":"philgyford",
+        "name":"Phil Gyford",
+        "indices":[
+          0,
+          11
+        ]
+      }
+    ],
+    "media":[
+
+    ],
+    "hashtags":[
+
+    ],
+    "symbols":[
+
+    ],
+    "urls":[
+
+    ]
+  },
+  "is_quote_status":false,
+  "possibly_sensitive":false
+}

tests/tweets/retweet.json

@@ -0,0 +1,91 @@
+{
+  "coordinates":null,
+  "source":"web",
+  "in_reply_to_user_id_str":null,
+  "truncated":false,
+  "in_reply_to_user_id":null,
+  "favorite_count":0,
+  "user":{
+    "name":"Phil Gyford Test",
+    "screen_name":"philgyfordtest"
+  },
+  "favorited":false,
+  "retweeted_status":{
+    "coordinates":null,
+    "source":"<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>",
+    "in_reply_to_user_id_str":null,
+    "truncated":false,
+    "in_reply_to_user_id":null,
+    "favorite_count":21,
+    "user":{
+      "name":"Samuel Pepys",
+      "screen_name":"samuelpepys"
+    },
+    "favorited":false,
+    "id":917459832885653506,
+    "contributors":null,
+    "in_reply_to_screen_name":null,
+    "geo":null,
+    "in_reply_to_status_id_str":null,
+    "id_str":"917459832885653506",
+    "entities":{
+      "hashtags":[
+
+      ],
+      "symbols":[
+
+      ],
+      "user_mentions":[
+
+      ],
+      "urls":[
+
+      ]
+    },
+    "in_reply_to_status_id":null,
+    "text":"My aunt and uncle in a very ill humour one with another, but I made shift with much ado to keep them from scolding.",
+    "retweet_count":3,
+    "place":null,
+    "lang":"en",
+    "created_at":"Mon Oct 09 18:40:44 +0000 2017",
+    "is_quote_status":false,
+    "retweeted":false
+  },
+  "id":917712989649801216,
+  "contributors":null,
+  "in_reply_to_screen_name":null,
+  "geo":null,
+  "in_reply_to_status_id_str":null,
+  "id_str":"917712989649801216",
+  "entities":{
+    "hashtags":[
+
+    ],
+    "symbols":[
+
+    ],
+    "user_mentions":[
+      {
+        "id_str":"14475268",
+        "indices":[
+          3,
+          15
+        ],
+        "name":"Samuel Pepys",
+        "id":14475268,
+        "screen_name":"samuelpepys"
+      }
+    ],
+    "urls":[
+
+    ]
+  },
+  "in_reply_to_status_id":null,
+  "text":"RT @samuelpepys: My aunt and uncle in a very ill humour one with another, but I made shift with much ado to keep them from scolding.",
+  "retweet_count":3,
+  "place":null,
+  "lang":"en",
+  "created_at":"Tue Oct 10 11:26:41 +0000 2017",
+  "is_quote_status":false,
+  "retweeted":false
+}

tests/tweets/symbols.json

@@ -0,0 +1,61 @@
+{
+  "text":"Some symbols: $AAPL and $PEP and $ANOTHER and $A.",
+  "contributors":null,
+  "geo":null,
+  "favorited":true,
+  "in_reply_to_user_id_str":null,
+  "user":{
+    "screen_name":"philgyfordtest",
+    "name":"Phil Gyford Test"
+  },
+  "in_reply_to_user_id":null,
+  "retweeted":false,
+  "coordinates":null,
+  "place":null,
+  "in_reply_to_status_id":null,
+  "lang":"en",
+  "in_reply_to_status_id_str":null,
+  "truncated":false,
+  "retweet_count":0,
+  "is_quote_status":false,
+  "id":662694880657989632,
+  "id_str":"662694880657989632",
+  "in_reply_to_screen_name":null,
+  "favorite_count":1,
+  "entities":{
+    "hashtags":[
+
+    ],
+    "user_mentions":[
+
+    ],
+    "symbols":[
+      {
+        "indices":[
+          14,
+          19
+        ],
+        "text":"AAPL"
+      },
+      {
+        "indices":[
+          24,
+          28
+        ],
+        "text":"PEP"
+      },
+      {
+        "indices":[
+          46,
+          48
+        ],
+        "text":"A"
+      }
+    ],
+    "urls":[
+
+    ]
+  },
+  "created_at":"Fri Nov 06 18:15:46 +0000 2015",
+  "source":"<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>"
+}

twython/__init__.py

@@ -11,14 +11,15 @@ Twython
 
 Twython is a library for Python that wraps the Twitter API.
 
-It aims to abstract away all the API endpoints, so that additions to the library
+It aims to abstract away all the API endpoints, so that
-and/or the Twitter API won't cause any overall problems.
+additions to the library and/or the Twitter API won't
+cause any overall problems.
 
 Questions, comments? ryan@venodesigns.net
 """
 
 __author__ = 'Ryan McGrath <ryan@venodesigns.net>'
-__version__ = '3.1.2'
+__version__ = '3.9.1'
 
 from .api import Twython
 from .streaming import TwythonStreamer

twython/advisory.py

@@ -15,7 +15,8 @@ only TwythonDeprecationWarnings.
 
 
 class TwythonDeprecationWarning(DeprecationWarning):
-    """Custom DeprecationWarning to be raised when methods/variables are being deprecated in Twython.
+    """Custom DeprecationWarning to be raised when methods/variables
-    Python 2.7 > ignores DeprecationWarning so we want to specifcally bubble up ONLY Twython Deprecation Warnings
+    are being deprecated in Twython. Python 2.7 > ignores DeprecationWarning
+    so we want to specifically bubble up ONLY Twython Deprecation Warnings
     """
     pass

twython/api.py

@@ -9,6 +9,10 @@ Twitter Authentication, and miscellaneous methods that are useful when
 dealing with the Twitter API
 """
 
+from __future__ import generator_stop
+import warnings
+import re
+
 import requests
 from requests.auth import HTTPBasicAuth
 from requests_oauthlib import OAuth1, OAuth2
@@ -16,40 +20,51 @@ from requests_oauthlib import OAuth1, OAuth2
 from . import __version__
 from .advisory import TwythonDeprecationWarning
 from .compat import json, urlencode, parse_qsl, quote_plus, str, is_py2
+from .compat import urlsplit
 from .endpoints import EndpointsMixin
 from .exceptions import TwythonError, TwythonAuthError, TwythonRateLimitError
 from .helpers import _transparent_params
 
-import warnings
-
 warnings.simplefilter('always', TwythonDeprecationWarning)  # For Python 2.7 >
 
 
 class Twython(EndpointsMixin, object):
     def __init__(self, app_key=None, app_secret=None, oauth_token=None,
-                 oauth_token_secret=None, access_token=None, token_type='bearer',
+                 oauth_token_secret=None, access_token=None,
-                 oauth_version=1, api_version='1.1', client_args=None, auth_endpoint='authenticate'):
+                 token_type='bearer', oauth_version=1, api_version='1.1',
-        """Instantiates an instance of Twython. Takes optional parameters for authentication and such (see below).
+                 client_args=None, auth_endpoint='authenticate'):
+        """Instantiates an instance of Twython. Takes optional parameters for
+        authentication and such (see below).
 
         :param app_key: (optional) Your applications key
         :param app_secret: (optional) Your applications secret key
-        :param oauth_token: (optional) When using **OAuth 1**, combined with oauth_token_secret to make authenticated calls
+        :param oauth_token: (optional) When using **OAuth 1**, combined with
-        :param oauth_token_secret: (optional) When using **OAuth 1** combined with oauth_token to make authenticated calls
+        oauth_token_secret to make authenticated calls
-        :param access_token: (optional) When using **OAuth 2**, provide a valid access token if you have one
+        :param oauth_token_secret: (optional) When using **OAuth 1** combined
-        :param token_type: (optional) When using **OAuth 2**, provide your token type. Default: bearer
+        with oauth_token to make authenticated calls
-        :param oauth_version: (optional) Choose which OAuth version to use. Default: 1
+        :param access_token: (optional) When using **OAuth 2**, provide a
-        :param api_version: (optional) Choose which Twitter API version to use. Default: 1.1
+        valid access token if you have one
-
+        :param token_type: (optional) When using **OAuth 2**, provide your
-        :param client_args: (optional) Accepts some requests Session parameters and some requests Request parameters.
+        token type. Default: bearer
-                            See http://docs.python-requests.org/en/latest/api/#sessionapi and requests section below it for details.
+        :param oauth_version: (optional) Choose which OAuth version to use.
-                            [ex. headers, proxies, verify(SSL verification)]
+        Default: 1
-        :param auth_endpoint: (optional) Lets you select which authentication endpoint will use your application.
+        :param api_version: (optional) Choose which Twitter API version to
-                              This will allow the application to have DM access if the endpoint is 'authorize'.
+        use. Default: 1.1
-                              Default: authenticate.
 
+        :param client_args: (optional) Accepts some requests Session parameters
+        and some requests Request parameters.
+              See http://docs.python-requests.org/en/latest/api/#sessionapi
+              and requests section below it for details.
+              [ex. headers, proxies, verify(SSL verification)]
+        :param auth_endpoint: (optional) Lets you select which authentication
+        endpoint will use your application.
+              This will allow the application to have DM access
+              if the endpoint is 'authorize'.
+                Default: authenticate.
         """
 
-        # API urls, OAuth urls and API version; needed for hitting that there API.
+        # API urls, OAuth urls and API version; needed for hitting that there
+        # API.
         self.api_version = api_version
         self.api_url = 'https://api.twitter.com/%s'
 
@@ -75,30 +90,29 @@ class Twython(EndpointsMixin, object):
 
         self.client_args = client_args or {}
         default_headers = {'User-Agent': 'Twython v' + __version__}
-        if not 'headers' in self.client_args:
+        if 'headers' not in self.client_args:
             # If they didn't set any headers, set our defaults for them
             self.client_args['headers'] = default_headers
         elif 'User-Agent' not in self.client_args['headers']:
-            # If they set headers, but didn't include User-Agent.. set it for them
+            # If they set headers, but didn't include User-Agent.. set
+            # it for them
             self.client_args['headers'].update(default_headers)
 
         # Generate OAuth authentication object for the request
         # If no keys/tokens are passed to __init__, auth=None allows for
-        # unauthenticated requests, although I think all v1.1 requests need auth
+        # unauthenticated requests, although I think all v1.1 requests
+        # need auth
         auth = None
         if oauth_version == 1:
             # User Authentication is through OAuth 1
-            if self.app_key is not None and self.app_secret is not None and \
+            if self.app_key is not None and self.app_secret is not None:
-               self.oauth_token is None and self.oauth_token_secret is None:
-                auth = OAuth1(self.app_key, self.app_secret)
-
-            if self.app_key is not None and self.app_secret is not None and \
-               self.oauth_token is not None and self.oauth_token_secret is not None:
                 auth = OAuth1(self.app_key, self.app_secret,
-                              self.oauth_token, self.oauth_token_secret)
+                                self.oauth_token, self.oauth_token_secret)
+
         elif oauth_version == 2 and self.access_token:
             # Application Authentication is through OAuth 2
-            token = {'token_type': token_type, 'access_token': self.access_token}
+            token = {'token_type': token_type,
+                     'access_token': self.access_token}
             auth = OAuth2(self.app_key, token=token)
 
         self.client = requests.Session()
@@ -109,22 +123,30 @@ class Twython(EndpointsMixin, object):
         # Never be used again.
         client_args_copy = self.client_args.copy()
         for k, v in client_args_copy.items():
-            if k in ('cert', 'headers', 'hooks', 'max_redirects', 'proxies'):
+            if k in ('cert', 'hooks', 'max_redirects', 'proxies'):
                 setattr(self.client, k, v)
                 self.client_args.pop(k)  # Pop, pop!
 
+        # Headers are always present, so we unconditionally pop them and merge
+        # them into the session headers.
+        self.client.headers.update(self.client_args.pop('headers'))
+
         self._last_call = None
 
     def __repr__(self):
         return '<Twython: %s>' % (self.app_key)
 
-    def _request(self, url, method='GET', params=None, api_call=None):
+    def _request(self, url, method='GET', params=None, api_call=None, json_encoded=False):
         """Internal request method"""
         method = method.lower()
         params = params or {}
 
         func = getattr(self.client, method)
-        params, files = _transparent_params(params)
+        if isinstance(params, dict) and json_encoded is False:
+            params, files = _transparent_params(params)
+        else:
+            params = params
+            files = list()
 
         requests_args = {}
         for k, v in self.client_args.items():
@@ -132,18 +154,22 @@ class Twython(EndpointsMixin, object):
             if k in ('timeout', 'allow_redirects', 'stream', 'verify'):
                 requests_args[k] = v
 
-        if method == 'get':
+        if method == 'get' or method == 'delete':
             requests_args['params'] = params
         else:
+            # Check for json_encoded so we will sent params as "data" or "json"
+            if json_encoded:
+                data_key = 'json'
+            else:
+                data_key = 'data'
             requests_args.update({
-                'data': params,
+                data_key: params,
                 'files': files,
             })
         try:
             response = func(url, **requests_args)
         except requests.RequestException as e:
             raise TwythonError(str(e))
-        content = response.content.decode('utf-8')
 
         # create stash for last function intel
         self._last_call = {
@@ -153,75 +179,98 @@ class Twython(EndpointsMixin, object):
             'headers': response.headers,
             'status_code': response.status_code,
             'url': response.url,
-            'content': content,
+            'content': response.text,
         }
 
-        #  Wrap the json loads in a try, and defer an error
+        # greater than 304 (not modified) is an error
-        #  Twitter will return invalid json with an error code in the headers
-        json_error = False
-        try:
-            try:
-                # try to get json
-                content = content.json()
-            except AttributeError:
-                # if unicode detected
-                content = json.loads(content)
-        except ValueError:
-            json_error = True
-            content = {}
-
         if response.status_code > 304:
-            # If there is no error message, use a default.
+            error_message = self._get_error_message(response)
-            errors = content.get('errors',
-                                 [{'message': 'An error occurred processing your request.'}])
-            if errors and isinstance(errors, list):
-                error_message = errors[0]['message']
-            else:
-                error_message = errors  # pragma: no cover
             self._last_call['api_error'] = error_message
 
             ExceptionType = TwythonError
             if response.status_code == 429:
-                # Twitter API 1.1, always return 429 when rate limit is exceeded
+                # Twitter API 1.1, always return 429 when
-                ExceptionType = TwythonRateLimitError  # pragma: no cover
+                # rate limit is exceeded
-            elif response.status_code == 401 or 'Bad Authentication data' in error_message:
+                ExceptionType = TwythonRateLimitError
+            elif response.status_code == 401 or 'Bad Authentication data' \
+                    in error_message:
                 # Twitter API 1.1, returns a 401 Unauthorized or
-                # a 400 "Bad Authentication data" for invalid/expired app keys/user tokens
+                # a 400 "Bad Authentication data" for invalid/expired
+                # app keys/user tokens
                 ExceptionType = TwythonAuthError
 
-            raise ExceptionType(error_message,
+            raise ExceptionType(
-                                error_code=response.status_code,
+                error_message,
-                                retry_after=response.headers.get('retry-after'))
+                error_code=response.status_code,
-
+                retry_after=response.headers.get('X-Rate-Limit-Reset'))
-        # if we have a json error here, then it's not an official Twitter API error
+        content = ''
-        if json_error and not response.status_code in (200, 201, 202):  # pragma: no cover
+        try:
-            raise TwythonError('Response was not valid JSON, unable to decode.')
+            if response.status_code == 204:
+                content = response.content
+            else:
+                content = response.json()
+        except ValueError:
+            if response.content != '':
+                raise TwythonError('Response was not valid JSON. \
+                                   Unable to decode.')
 
         return content
 
-    def request(self, endpoint, method='GET', params=None, version='1.1'):
+    def _get_error_message(self, response):
+        """Parse and return the first error message"""
+
+        error_message = 'An error occurred processing your request.'
+        try:
+            content = response.json()
+            # {"errors":[{"code":34,"message":"Sorry,
+            # that page does not exist"}]}
+            error_message = content['errors'][0]['message']
+        except TypeError:
+            error_message = content['errors']
+        except ValueError:
+            # bad json data from Twitter for an error
+            pass
+        except (KeyError, IndexError):
+            # missing data so fallback to default message
+            pass
+
+        return error_message
+
+    def request(self, endpoint, method='GET', params=None, version='1.1', json_encoded=False):
         """Return dict of response received from Twitter's API
 
-        :param endpoint: (required) Full url or Twitter API endpoint (e.g. search/tweets)
+        :param endpoint: (required) Full url or Twitter API endpoint
+                         (e.g. search/tweets)
         :type endpoint: string
-        :param method: (optional) Method of accessing data, either GET or POST. (default GET)
+        :param method: (optional) Method of accessing data, either
+                       GET, POST or DELETE. (default GET)
         :type method: string
-        :param params: (optional) Dict of parameters (if any) accepted the by Twitter API endpoint you are trying to access (default None)
+        :param params: (optional) Dict of parameters (if any) accepted
+                       the by Twitter API endpoint you are trying to
+                       access (default None)
         :type params: dict or None
-        :param version: (optional) Twitter API version to access (default 1.1)
+        :param version: (optional) Twitter API version to access
+                        (default 1.1)
         :type version: string
+        :param json_encoded: (optional) Flag to indicate if this method should send data encoded as json
+                        (default False)
+        :type json_encoded: bool
 
         :rtype: dict
         """
 
+        if endpoint.startswith('http://'):
+            raise TwythonError('api.twitter.com is restricted to SSL/TLS traffic.')
+
         # In case they want to pass a full Twitter URL
         # i.e. https://api.twitter.com/1.1/search/tweets.json
-        if endpoint.startswith('http://') or endpoint.startswith('https://'):
+        if endpoint.startswith('https://'):
             url = endpoint
         else:
             url = '%s/%s.json' % (self.api_url % version, endpoint)
 
-        content = self._request(url, method=method, params=params, api_call=url)
+        content = self._request(url, method=method, params=params,
+                                api_call=url, json_encoded=json_encoded)
 
         return content
 
@@ -229,15 +278,20 @@ class Twython(EndpointsMixin, object):
         """Shortcut for GET requests via :class:`request`"""
         return self.request(endpoint, params=params, version=version)
 
-    def post(self, endpoint, params=None, version='1.1'):
+    def post(self, endpoint, params=None, version='1.1', json_encoded=False):
         """Shortcut for POST requests via :class:`request`"""
-        return self.request(endpoint, 'POST', params=params, version=version)
+        return self.request(endpoint, 'POST', params=params, version=version, json_encoded=json_encoded)
+
+    def delete(self, endpoint, params=None, version='1.1', json_encoded=False):
+        """Shortcut for delete requests via :class:`request`"""
+        return self.request(endpoint, 'DELETE', params=params, version=version, json_encoded=json_encoded)
 
     def get_lastfunction_header(self, header, default_return_value=None):
         """Returns a specific header from the last API call
         This will return None if the header is not present
 
-        :param header: (required) The name of the header you want to get the value of
+        :param header: (required) The name of the header you want to get
+                       the value of
 
         Most useful for the following header information:
             x-rate-limit-limit,
@@ -247,21 +301,31 @@ class Twython(EndpointsMixin, object):
 
         """
         if self._last_call is None:
-            raise TwythonError('This function must be called after an API call.  It delivers header information.')
+            raise TwythonError('This function must be called after an API call. \
+                               It delivers header information.')
 
         return self._last_call['headers'].get(header, default_return_value)
 
-    def get_authentication_tokens(self, callback_url=None, force_login=False, screen_name=''):
+    def get_authentication_tokens(self, callback_url=None, force_login=False,
-        """Returns a dict including an authorization URL, ``auth_url``, to direct a user to
+                                  screen_name=''):
+        """Returns a dict including an authorization URL, ``auth_url``, to
+           direct a user to
 
-        :param callback_url: (optional) Url the user is returned to after they authorize your app (web clients only)
+        :param callback_url: (optional) Url the user is returned to after
-        :param force_login: (optional) Forces the user to enter their credentials to ensure the correct users account is authorized.
+                             they authorize your app (web clients only)
-        :param screen_name: (optional) If forced_login is set OR user is not currently logged in, Prefills the username input box of the OAuth login screen with the given value
+        :param force_login: (optional) Forces the user to enter their
+                            credentials to ensure the correct users
+                            account is authorized.
+        :param screen_name: (optional) If forced_login is set OR user is
+                            not currently logged in, Prefills the username
+                            input box of the OAuth login screen with the
+                            given value
 
         :rtype: dict
         """
         if self.oauth_version != 1:
-            raise TwythonError('This method can only be called when your OAuth version is 1.0.')
+            raise TwythonError('This method can only be called when your \
+                               OAuth version is 1.0.')
 
         request_args = {}
         if callback_url:
@@ -269,15 +333,18 @@ class Twython(EndpointsMixin, object):
         response = self.client.get(self.request_token_url, params=request_args)
 
         if response.status_code == 401:
-            raise TwythonAuthError(response.content, error_code=response.status_code)
+            raise TwythonAuthError(response.content,
+                                   error_code=response.status_code)
         elif response.status_code != 200:
-            raise TwythonError(response.content, error_code=response.status_code)
+            raise TwythonError(response.content,
+                               error_code=response.status_code)
 
         request_tokens = dict(parse_qsl(response.content.decode('utf-8')))
         if not request_tokens:
             raise TwythonError('Unable to decode request tokens.')
 
-        oauth_callback_confirmed = request_tokens.get('oauth_callback_confirmed') == 'true'
+        oauth_callback_confirmed = request_tokens.get('oauth_callback_confirmed') \
+            == 'true'
 
         auth_url_params = {
             'oauth_token': request_tokens['oauth_token'],
@@ -293,21 +360,28 @@ class Twython(EndpointsMixin, object):
         if callback_url and not oauth_callback_confirmed:
             auth_url_params['oauth_callback'] = self.callback_url
 
-        request_tokens['auth_url'] = self.authenticate_url + '?' + urlencode(auth_url_params)
+        request_tokens['auth_url'] = self.authenticate_url + \
+            '?' + urlencode(auth_url_params)
 
         return request_tokens
 
     def get_authorized_tokens(self, oauth_verifier):
-        """Returns a dict of authorized tokens after they go through the :class:`get_authentication_tokens` phase.
+        """Returns a dict of authorized tokens after they go through the
+        :class:`get_authentication_tokens` phase.
 
-        :param oauth_verifier: (required) The oauth_verifier (or a.k.a PIN for non web apps) retrieved from the callback url querystring
+        :param oauth_verifier: (required) The oauth_verifier (or a.k.a PIN
+        for non web apps) retrieved from the callback url querystring
         :rtype: dict
 
         """
         if self.oauth_version != 1:
-            raise TwythonError('This method can only be called when your OAuth version is 1.0.')
+            raise TwythonError('This method can only be called when your \
+                               OAuth version is 1.0.')
 
-        response = self.client.get(self.access_token_url, params={'oauth_verifier': oauth_verifier}, headers={'Content-Type': 'application/json'})
+        response = self.client.get(self.access_token_url,
+                                   params={'oauth_verifier': oauth_verifier},
+                                   headers={'Content-Type': 'application/\
+                                   json'})
 
         if response.status_code == 401:
             try:
@@ -320,7 +394,8 @@ class Twython(EndpointsMixin, object):
             except ValueError:
                 content = {}
 
-            raise TwythonError(content.get('error', 'Invalid / expired Token'), error_code=response.status_code)
+            raise TwythonError(content.get('error', 'Invalid / expired To \
+            ken'), error_code=response.status_code)
 
         authorized_tokens = dict(parse_qsl(response.content.decode('utf-8')))
         if not authorized_tokens:
@@ -329,12 +404,14 @@ class Twython(EndpointsMixin, object):
         return authorized_tokens  # pragma: no cover
 
     def obtain_access_token(self):
-        """Returns an OAuth 2 access token to make OAuth 2 authenticated read-only calls.
+        """Returns an OAuth 2 access token to make OAuth 2 authenticated
+        read-only calls.
 
         :rtype: string
         """
         if self.oauth_version != 2:
-            raise TwythonError('This method can only be called when your OAuth version is 2.0.')
+            raise TwythonError('This method can only be called when your \
+                               OAuth version is 2.0.')
 
         data = {'grant_type': 'client_credentials'}
         basic_auth = HTTPBasicAuth(self.app_key, self.app_secret)
@@ -354,10 +431,12 @@ class Twython(EndpointsMixin, object):
 
     @staticmethod
     def construct_api_url(api_url, **params):
-        """Construct a Twitter API url, encoded, with parameters
+        r"""Construct a Twitter API url, encoded, with parameters
 
-        :param api_url: URL of the Twitter API endpoint you are attempting to construct
+        :param api_url: URL of the Twitter API endpoint you are attempting
-        :param \*\*params: Parameters that are accepted by Twitter for the endpoint you're requesting
+        to construct
+        :param \*\*params: Parameters that are accepted by Twitter for the
+        endpoint you're requesting
         :rtype: string
 
         Usage::
@@ -366,7 +445,8 @@ class Twython(EndpointsMixin, object):
           >>> twitter = Twython()
 
           >>> api_url = 'https://api.twitter.com/1.1/search/tweets.json'
-          >>> constructed_url = twitter.construct_api_url(api_url, q='python', result_type='popular')
+          >>> constructed_url = twitter.construct_api_url(api_url, q='python',
+          result_type='popular')
           >>> print constructed_url
           https://api.twitter.com/1.1/search/tweets.json?q=python&result_type=popular
 
@@ -382,64 +462,86 @@ class Twython(EndpointsMixin, object):
 
     def search_gen(self, search_query, **params):  # pragma: no cover
         warnings.warn(
-            'This method is deprecated. You should use Twython.cursor instead. [eg. Twython.cursor(Twython.search, q=\'your_query\')]',
+            'This method is deprecated. You should use Twython.cursor instead. \
+            [eg. Twython.cursor(Twython.search, q=\'your_query\')]',
             TwythonDeprecationWarning,
             stacklevel=2
         )
         return self.cursor(self.search, q=search_query, **params)
 
-    def cursor(self, function, **params):
+    def cursor(self, function, return_pages=False, **params):
-        """Returns a generator for results that match a specified query.
+        r"""Returns a generator for results that match a specified query.
 
-        :param function: Instance of a Twython function (Twython.get_home_timeline, Twython.search)
+        :param function: Instance of a Twython function
-        :param \*\*params: Extra parameters to send with your request (usually parameters excepted by the Twitter API endpoint)
+        (Twython.get_home_timeline, Twython.search)
+        :param \*\*params: Extra parameters to send with your request
+        (usually parameters accepted by the Twitter API endpoint)
         :rtype: generator
 
         Usage::
 
           >>> from twython import Twython
-          >>> twitter = Twython(APP_KEY, APP_SECRET, OAUTH_TOKEN, OAUTH_TOKEN_SECRET)
+          >>> twitter = Twython(APP_KEY, APP_SECRET, OAUTH_TOKEN,
+          OAUTH_TOKEN_SECRET)
 
           >>> results = twitter.cursor(twitter.search, q='python')
           >>> for result in results:
           >>>   print result
 
         """
+        if not callable(function):
+            raise TypeError('.cursor() takes a Twython function as its first \
+                            argument. Did you provide the result of a \
+                            function call?')
+
         if not hasattr(function, 'iter_mode'):
-            raise TwythonError('Unable to create generator for Twython method "%s"' % function.__name__)
+            raise TwythonError('Unable to create generator for Twython \
+                               method "%s"' % function.__name__)
 
-        content = function(**params)
+        while True:
+            content = function(**params)
 
-        if not content:
+            if not content:
-            raise StopIteration
+                return
 
-        if hasattr(function, 'iter_key'):
+            if hasattr(function, 'iter_key'):
-            results = content.get(function.iter_key)
+                results = content.get(function.iter_key)
-        else:
+            else:
-            results = content
+                results = content
 
-        for result in results:
+            if return_pages:
-            yield result
+                yield results
+            else:
+                for result in results:
+                    yield result
 
-        if function.iter_mode == 'cursor' and content['next_cursor_str'] == '0':
+            if function.iter_mode == 'cursor' and \
-            raise StopIteration
+               content['next_cursor_str'] == '0':
+                return
 
-        try:
+            try:
-            if function.iter_mode == 'id':
+                if function.iter_mode == 'id':
-                if not 'max_id' in params:
+                    # Set max_id in params to one less than lowest tweet id
-                    # Add 1 to the id because since_id and max_id are inclusive
                     if hasattr(function, 'iter_metadata'):
-                        since_id = content[function.iter_metadata].get('since_id_str')
+                        # Get supplied next max_id
+                        metadata = content.get(function.iter_metadata)
+                        if 'next_results' in metadata:
+                            next_results = urlsplit(metadata['next_results'])
+                            params = dict(parse_qsl(next_results.query))
+                        else:
+                            # No more results
+                            return
                     else:
-                        since_id = content[0]['id_str']
+                        # Twitter gives tweets in reverse chronological order:
-                    params['since_id'] = (int(since_id) - 1)
+                        params['max_id'] = str(int(content[-1]['id_str']) - 1)
-            elif function.iter_mode == 'cursor':
+                elif function.iter_mode == 'cursor':
-                params['cursor'] = content['next_cursor_str']
+                    params['cursor'] = content['next_cursor_str']
-        except (TypeError, ValueError):  # pragma: no cover
+            except (TypeError, ValueError):  # pragma: no cover
-            raise TwythonError('Unable to generate next page of search results, `page` is not a number.')
+                raise TwythonError('Unable to generate next page of search \
-
+                                   results, `page` is not a number.')
-        for result in self.cursor(function, **params):
+            except (KeyError, AttributeError):  #pragma no cover
-            yield result
+                raise TwythonError('Unable to generate next page of search \
+                                   results, content has unexpected structure.')
 
     @staticmethod
     def unicode2utf8(text):
@@ -457,41 +559,115 @@ class Twython(EndpointsMixin, object):
         return str(text)
 
     @staticmethod
-    def html_for_tweet(tweet, use_display_url=True, use_expanded_url=False):
+    def html_for_tweet(tweet, use_display_url=True, use_expanded_url=False, expand_quoted_status=False):
-        """Return HTML for a tweet (urls, mentions, hashtags replaced with links)
+        """Return HTML for a tweet (urls, mentions, hashtags, symbols replaced with links)
 
         :param tweet: Tweet object from received from Twitter API
-        :param use_display_url: Use display URL to represent link (ex. google.com, github.com). Default: True
+        :param use_display_url: Use display URL to represent link
-        :param use_expanded_url: Use expanded URL to represent link (e.g. http://google.com). Default False
+        (ex. google.com, github.com). Default: True
+        :param use_expanded_url: Use expanded URL to represent link
+        (e.g. http://google.com). Default False
 
         If use_expanded_url is True, it overrides use_display_url.
-        If use_display_url and use_expanded_url is False, short url will be used (t.co/xxxxx)
+        If use_display_url and use_expanded_url is False, short url will
+        be used (t.co/xxxxx)
 
         """
         if 'retweeted_status' in tweet:
             tweet = tweet['retweeted_status']
 
+        if 'extended_tweet' in tweet:
+            tweet = tweet['extended_tweet']
+
+        orig_tweet_text = tweet.get('full_text') or tweet['text']
+
+        display_text_range = tweet.get('display_text_range') or [0, len(orig_tweet_text)]
+        display_text_start, display_text_end = display_text_range[0], display_text_range[1]
+        display_text = orig_tweet_text[display_text_start:display_text_end]
+        prefix_text = orig_tweet_text[0:display_text_start]
+        suffix_text = orig_tweet_text[display_text_end:len(orig_tweet_text)]
+
         if 'entities' in tweet:
-            text = tweet['text']
+            # We'll put all the bits of replacement HTML and their starts/ends
-            entities = tweet['entities']
+            # in this list:
+            entities = []
 
             # Mentions
-            for entity in entities['user_mentions']:
+            if 'user_mentions' in tweet['entities']:
-                start, end = entity['indices'][0], entity['indices'][1]
+                for entity in tweet['entities']['user_mentions']:
+                    temp = {}
+                    temp['start'] = entity['indices'][0]
+                    temp['end'] = entity['indices'][1]
 
-                mention_html = '<a href="https://twitter.com/%(screen_name)s" class="twython-mention">@%(screen_name)s</a>'
+                    mention_html = '<a href="https://twitter.com/%(screen_name)s" class="twython-mention">@%(screen_name)s</a>' % {'screen_name': entity['screen_name']}
-                text = text.replace(tweet['text'][start:end], mention_html % {'screen_name': entity['screen_name']})
+
+                    if display_text_start <= temp['start'] <= display_text_end:
+                        temp['replacement'] = mention_html
+                        temp['start'] -= display_text_start
+                        temp['end'] -= display_text_start
+                        entities.append(temp)
+                    else:
+                        # Make the '@username' at the start, before
+                        # display_text, into a link:
+                        sub_expr = r'(?<!>)' + orig_tweet_text[temp['start']:temp['end']] + '(?!</a>)'
+                        prefix_text = re.sub(sub_expr, mention_html, prefix_text)
 
             # Hashtags
-            for entity in entities['hashtags']:
+            if 'hashtags' in tweet['entities']:
-                start, end = entity['indices'][0], entity['indices'][1]
+                for entity in tweet['entities']['hashtags']:
+                    temp = {}
+                    temp['start'] = entity['indices'][0] - display_text_start
+                    temp['end'] = entity['indices'][1] - display_text_start
 
-                hashtag_html = '<a href="https://twitter.com/search?q=%%23%(hashtag)s" class="twython-hashtag">#%(hashtag)s</a>'
+                    url_html = '<a href="https://twitter.com/search?q=%%23%(hashtag)s" class="twython-hashtag">#%(hashtag)s</a>' % {'hashtag': entity['text']}
-                text = text.replace(tweet['text'][start:end], hashtag_html % {'hashtag': entity['text']})
+
+                    temp['replacement'] = url_html
+                    entities.append(temp)
+
+            # Symbols
+            if 'symbols' in tweet['entities']:
+                for entity in tweet['entities']['symbols']:
+                    temp = {}
+                    temp['start'] = entity['indices'][0] - display_text_start
+                    temp['end'] = entity['indices'][1] - display_text_start
+
+                    url_html = '<a href="https://twitter.com/search?q=%%24%(symbol)s" class="twython-symbol">$%(symbol)s</a>' % {'symbol': entity['text']}
+
+                    temp['replacement'] = url_html
+                    entities.append(temp)
+
+            # URLs
+            if 'urls' in tweet['entities']:
+                for entity in tweet['entities']['urls']:
+                    temp = {}
+                    temp['start'] = entity['indices'][0] - display_text_start
+                    temp['end'] = entity['indices'][1] - display_text_start
+
+                    if use_display_url and entity.get('display_url') and not use_expanded_url:
+                        shown_url = entity['display_url']
+                    elif use_expanded_url and entity.get('expanded_url'):
+                        shown_url = entity['expanded_url']
+                    else:
+                        shown_url = entity['url']
+
+                    url_html = '<a href="%s" class="twython-url">%s</a>' % (entity['url'], shown_url)
+
+                    if display_text_start <= temp['start'] <= display_text_end:
+                        temp['replacement'] = url_html
+                        entities.append(temp)
+                    else:
+                        suffix_text = suffix_text.replace(orig_tweet_text[temp['start']:temp['end']], url_html)
+
+            if 'media' in tweet['entities'] and len(tweet['entities']['media']) > 0:
+                # We just link to the overall URL for the tweet's media,
+                # rather than to each individual item.
+                # So, we get the URL from the first media item:
+                entity = tweet['entities']['media'][0]
+
+                temp = {}
+                temp['start'] = entity['indices'][0]
+                temp['end'] = entity['indices'][1]
 
-            # Urls
-            for entity in entities['urls']:
-                start, end = entity['indices'][0], entity['indices'][1]
                 if use_display_url and entity.get('display_url') and not use_expanded_url:
                     shown_url = entity['display_url']
                 elif use_expanded_url and entity.get('expanded_url'):
@@ -499,7 +675,35 @@ class Twython(EndpointsMixin, object):
                 else:
                     shown_url = entity['url']
 
-                url_html = '<a href="%s" class="twython-url">%s</a>'
+                url_html = '<a href="%s" class="twython-media">%s</a>' % (entity['url'], shown_url)
-                text = text.replace(tweet['text'][start:end], url_html % (entity['url'], shown_url))
 
-        return text
+                if display_text_start <= temp['start'] <= display_text_end:
+                    temp['replacement'] = url_html
+                    entities.append(temp)
+                else:
+                    suffix_text = suffix_text.replace(orig_tweet_text[temp['start']:temp['end']], url_html)
+
+            # Now do all the replacements, starting from the end, so that the
+            # start/end indices still work:
+            for entity in sorted(entities, key=lambda e: e['start'], reverse=True):
+                display_text = display_text[0:entity['start']] + entity['replacement'] + display_text[entity['end']:]
+
+        quote_text = ''
+        if expand_quoted_status and tweet.get('is_quote_status') and tweet.get('quoted_status'):
+            quoted_status = tweet['quoted_status']
+            quote_text += '<blockquote class="twython-quote">%(quote)s<cite><a href="%(quote_tweet_link)s">' \
+                    '<span class="twython-quote-user-name">%(quote_user_name)s</span>' \
+                    '<span class="twython-quote-user-screenname">@%(quote_user_screen_name)s</span></a>' \
+                    '</cite></blockquote>' % \
+                    {'quote': Twython.html_for_tweet(quoted_status, use_display_url, use_expanded_url, False),
+                     'quote_tweet_link': 'https://twitter.com/%s/status/%s' %
+                                         (quoted_status['user']['screen_name'], quoted_status['id_str']),
+                     'quote_user_name': quoted_status['user']['name'],
+                     'quote_user_screen_name': quoted_status['user']['screen_name']}
+
+        return '%(prefix)s%(display)s%(suffix)s%(quote)s' % {
+            'prefix': '<span class="twython-tweet-prefix">%s</span>' % prefix_text if prefix_text else '',
+            'display': display_text,
+            'suffix': '<span class="twython-tweet-suffix">%s</span>' % suffix_text if suffix_text else '',
+            'quote': quote_text
+        }

twython/compat.py

@@ -25,7 +25,7 @@ except ImportError:
 
 if is_py2:
     from urllib import urlencode, quote_plus
-    from urlparse import parse_qsl
+    from urlparse import parse_qsl, urlsplit
 
     str = unicode
     basestring = basestring
@@ -33,7 +33,7 @@ if is_py2:
 
 
 elif is_py3:
-    from urllib.parse import urlencode, quote_plus, parse_qsl
+    from urllib.parse import urlencode, quote_plus, parse_qsl, urlsplit
 
     str = str
     basestring = (str, bytes)

twython/exceptions.py

@@ -53,6 +53,8 @@ class TwythonRateLimitError(TwythonError):  # pragma: no cover
             msg = '%s (Retry after %d seconds)' % (msg, retry_after)
         TwythonError.__init__(self, msg, error_code=error_code)
 
+        self.retry_after = retry_after
+
 
 class TwythonStreamError(TwythonError):
     """Raised when an invalid response from the Stream API is received"""

twython/helpers.py

@@ -28,7 +28,7 @@ def _transparent_params(_params):
             try:
                 params[k] = ','.join(v)
             except TypeError:
-                params[k] = ','.join(map(str,v))
+                params[k] = ','.join(map(str, v))
         else:
             continue  # pragma: no cover
     return params, files

twython/streaming/api.py

@@ -38,8 +38,11 @@ class TwythonStreamer(object):
                             retired
         :param retry_in: (optional) Amount of time (in secs) the previous
                          API call should be tried again
-        :param client_args: (optional) Accepts some requests Session parameters and some requests Request parameters.
+        :param client_args: (optional) Accepts some requests Session
-                            See http://docs.python-requests.org/en/latest/api/#sessionapi and requests section below it for details.
+                            parameters and some requests Request parameters.
+                            See
+                            http://docs.python-requests.org/en/latest/api/#sessionapi
+                            and requests section below it for details.
                             [ex. headers, proxies, verify(SSL verification)]
         :param handlers: (optional) Array of message types for which
                          corresponding handlers will be called
@@ -53,11 +56,12 @@ class TwythonStreamer(object):
 
         self.client_args = client_args or {}
         default_headers = {'User-Agent': 'Twython Streaming v' + __version__}
-        if not 'headers' in self.client_args:
+        if 'headers' not in self.client_args:
             # If they didn't set any headers, set our defaults for them
             self.client_args['headers'] = default_headers
         elif 'User-Agent' not in self.client_args['headers']:
-            # If they set headers, but didn't include User-Agent.. set it for them
+            # If they set headers, but didn't include User-Agent..
+            # set it for them
             self.client_args['headers'].update(default_headers)
         self.client_args['timeout'] = timeout
 
@@ -82,12 +86,11 @@ class TwythonStreamer(object):
         # Set up type methods
         StreamTypes = TwythonStreamerTypes(self)
         self.statuses = StreamTypes.statuses
-        self.user = StreamTypes.user
-        self.site = StreamTypes.site
 
         self.connected = False
 
-        self.handlers = handlers if handlers else ['delete', 'limit', 'disconnect']
+        self.handlers = handlers if handlers else \
+            ['delete', 'limit', 'disconnect']
 
         self.chunk_size = chunk_size
 
@@ -103,7 +106,8 @@ class TwythonStreamer(object):
         def _send(retry_counter):
             requests_args = {}
             for k, v in self.client_args.items():
-            # Maybe this should be set as a class variable and only done once?
+                # Maybe this should be set as a class
+                # variable and only done once?
                 if k in ('timeout', 'allow_redirects', 'verify'):
                     requests_args[k] = v
 
@@ -119,9 +123,10 @@ class TwythonStreamer(object):
                     self.on_timeout()
                 else:
                     if response.status_code != 200:
-                        self.on_error(response.status_code, response.content)
+                        self.on_error(response.status_code, response.content, response.headers)
 
-                    if self.retry_count and (self.retry_count - retry_counter) > 0:
+                    if self.retry_count and \
+                       (self.retry_count - retry_counter) > 0:
                         time.sleep(self.retry_in)
                         retry_counter += 1
                         _send(retry_counter)
@@ -140,13 +145,19 @@ class TwythonStreamer(object):
                             line = line.decode('utf-8')
                         data = json.loads(line)
                     except ValueError:  # pragma: no cover
-                        self.on_error(response.status_code, 'Unable to decode response, not valid JSON.')
+                        self.on_error(response.status_code,
+                                      'Unable to decode response, \
+                                      not valid JSON.')
                     else:
                         if self.on_success(data):  # pragma: no cover
                             for message_type in self.handlers:
                                 if message_type in data:
-                                    handler = getattr(self, 'on_' + message_type, None)
+                                    handler = getattr(self,
-                                    if handler and callable(handler) and not handler(data.get(message_type)):
+                                                      'on_' + message_type,
+                                                      None)
+                                    if handler \
+                                       and callable(handler) \
+                                       and not handler(data.get(message_type)):
                                         break
 
         response.close()
@@ -156,16 +167,16 @@ class TwythonStreamer(object):
         Returns True if other handlers for this message should be invoked.
 
         Feel free to override this to handle your streaming data how you
-        want it handled.
+        want it handled. See
-        See https://dev.twitter.com/docs/streaming-apis/messages for messages
+        https://developer.twitter.com/en/docs/tweets/filter-realtime/guides/streaming-message-types
-        sent along in stream responses.
+        for messages sent along in stream responses.
 
         :param data: data recieved from the stream
         :type data: dict
         """
         return True
 
-    def on_error(self, status_code, data):  # pragma: no cover
+    def on_error(self, status_code, data, headers=None):  # pragma: no cover
         """Called when stream returns non-200 status code
 
         Feel free to override this to handle your streaming data how you
@@ -176,6 +187,9 @@ class TwythonStreamer(object):
 
         :param data: Error message sent from stream
         :type data: dict
+
+        :param headers: Response headers sent from the stream (i.e. Retry-After)
+        :type headers: dict
         """
         return
 

twython/streaming/types.py

@@ -20,64 +20,46 @@ class TwythonStreamerTypes(object):
         self.streamer = streamer
         self.statuses = TwythonStreamerTypesStatuses(streamer)
 
-    def user(self, **params):
-        """Stream user
-
-        Accepted params found at:
-        https://dev.twitter.com/docs/api/1.1/get/user
-        """
-        url = 'https://userstream.twitter.com/%s/user.json' \
-              % self.streamer.api_version
-        self.streamer._request(url, params=params)
-
-    def site(self, **params):
-        """Stream site
-
-        Accepted params found at:
-        https://dev.twitter.com/docs/api/1.1/get/site
-        """
-        url = 'https://sitestream.twitter.com/%s/site.json' \
-              % self.streamer.api_version
-        self.streamer._request(url, params=params)
-
 
 class TwythonStreamerTypesStatuses(object):
     """Class for different statuses endpoints
 
-    Available so TwythonStreamer.statuses.filter() is available.
+    Available so :meth:`TwythonStreamer.statuses.filter()` is available.
-    Just a bit cleaner than TwythonStreamer.statuses_filter(),
+    Just a bit cleaner than :meth:`TwythonStreamer.statuses_filter()`,
-    statuses_sample(), etc. all being single methods in TwythonStreamer
+    :meth:`statuses_sample()`, etc. all being single methods in 
+    :class:`TwythonStreamer`.
 
     """
     def __init__(self, streamer):
         self.streamer = streamer
+        self.params = None
 
     def filter(self, **params):
-        """Stream statuses/filter
+        r"""Stream statuses/filter
 
         :param \*\*params: Parameters to send with your stream request
 
         Accepted params found at:
-        https://dev.twitter.com/docs/api/1.1/post/statuses/filter
+        https://developer.twitter.com/en/docs/tweets/filter-realtime/api-reference/post-statuses-filter
         """
         url = 'https://stream.twitter.com/%s/statuses/filter.json' \
               % self.streamer.api_version
         self.streamer._request(url, 'POST', params=params)
 
     def sample(self, **params):
-        """Stream statuses/sample
+        r"""Stream statuses/sample
 
         :param \*\*params: Parameters to send with your stream request
 
         Accepted params found at:
-        https://dev.twitter.com/docs/api/1.1/get/statuses/sample
+        https://developer.twitter.com/en/docs/tweets/sample-realtime/api-reference/get-statuses-sample
         """
         url = 'https://stream.twitter.com/%s/statuses/sample.json' \
               % self.streamer.api_version
         self.streamer._request(url, params=params)
 
     def firehose(self, **params):
-        """Stream statuses/firehose
+        r"""Stream statuses/firehose
 
         :param \*\*params: Parameters to send with your stream request
 
@@ -87,3 +69,20 @@ class TwythonStreamerTypesStatuses(object):
         url = 'https://stream.twitter.com/%s/statuses/firehose.json' \
               % self.streamer.api_version
         self.streamer._request(url, params=params)
+
+    def set_dynamic_filter(self, **params):
+        r"""Set/update statuses/filter
+
+        :param \*\*params: Parameters to send with your stream request
+
+        Accepted params found at:
+        https://developer.twitter.com/en/docs/tweets/filter-realtime/api-reference/post-statuses-filter
+        """
+        self.params = params
+
+    def dynamic_filter(self):
+        """Stream statuses/filter with dynamic parameters"""
+
+        url = 'https://stream.twitter.com/%s/statuses/filter.json' \
+              % self.streamer.api_version
+        self.streamer._request(url, 'POST', params=self.params)