Search API Limits (“Low Volume, incl. History” Searches)

The integration of Twitter’s Search or REST API into Viz Social is described in Twitter. Known volume or performance restrictions are:

• Twitter limits API access when more than 450 calls per interval of 15 minutes are made. With the default Viz Social setup, where Searches are repeated once a minute, this implies that the interface will be rate-limited by Twitter when more than 30 (=450/15) of these Searches are active on the platform at the same time. The effect of rate limiting is that Searches return no results until the 15-minute interval is over.

• Rate limiting kicks in even earlier for Searches that return a lot of hits, typically more than 100 hits/minute. Each page of 100 results after the first one requires a new request and thus effectively counts as a new Search.

• In reality, the effective maximum throughput strongly depends on how efficient the pages are filled. If all of them are full the theoretical maximum throughput is 50 Tweets/sec. If each of them just contains a single Tweet, then the effective throughput is only 0.5 Tweets/sec. In reality, the Twitter API rarely uses all available bandwidth, so +/- 5 Tweets/sec is more a typical number often observes as a maximum for this API.

• Each Search phrase individually must be less than 500 characters.

• At most 20 Twitter users can be monitored per Search.

The maximum holds per server for the combined volume of all active Twitter Searches using the Twitter Search API. In case these limits are too restrictive, we advise using Twitter’s Streaming API instead.

Streaming API Limits (“High Volume, no History” Searches)

The integration of Twitter’s Streaming API into Viz Social is described in Twitter. Known volume or performance restrictions are:

• Twitter limits the returned volume of Tweets over its Streaming API to roughly 1% of the momentary aggregated worldwide volume. This roughly translates into an average API maximum of 60 Tweets/sec on a normal day. During peaked, special events like the Oscars or the World Cup Finale, this may well increase to over a 100 Tweets/sec or more.

• Viz Social is able to harvest at a rate of 15-20 messages/sec from this API.

• The number of Search creations, modifications, deletions should be less than one per minute to prevent Twitter from temporarily suspending traffic.

• At any point in time, the total number of searched terms on the installation should not exceed 400 and the total number of monitored users on the installation should not exceed 5000.

The maximum holds per server for the combined volume of all active Searches using the Twitter Streaming API.

Filtered Firehose Limits

The integration of Twitter’s Firehose API into Viz Social is described in Twitter. It is guaranteed complete, independent of volume and rate.

Nevertheless, each installation with finite physical resources (CPU, memory, …) necessarily has its limits. When Tweets start to arrive at a faster rate than the CPU can process, the surplus will be queued up and, as a result, users will start noticing delays. This continues until the max queue size (dictated by the size of the system’s memory) is reached, after which the surplus is ignored because there’s simply no place anymore for them.

Hence, it is crucial that the installed hardware is in line with the expected traffic profile when every Tweet matters. To give some guidance:

• A standard Viz Social server can process 200 Tweets/sec without queues forming^*.

• When traffic levels are further increased to 1000 Tweets/sec, a standard virgin Viz Social server can withhold and queue up messages for at least 15 minutes^*.

• These numbers are affected adversely by:

  • Active Formats, particularly restricted Polls.

  • Different users logged in and active on the same Format/Search.

  • Many messages already stored on the platform (3-5M or more).

• Capacity figures can be further increased by scaling horizontally and/or vertically.

Auto Reply Limits

Twitter restricts the number of Tweets originating from any account to 2400 per day (imposed in 30 minute intervals) and this limit also applies to replies automatically generated by Viz Social. For certain cases, always explicitly agreed in advance, Twitter may be able to raise the limit well-beyond its default value. Support is able to address this matter via their Twitter contacts, so please contact your representative in case the standard rate is insufficient for your use cases.

API Limits

The integration of Instagram into Viz Social is described in Instagram. Instagram’s API allows Viz Social to run 83 Instagram Searches in parallel per server.

• Instagram limits API access when more than 5000 calls per hour are made. With the default Viz Social set-up where Searches are repeated once a minute, this implies that the interface will be rate-limited by Instagram when more than 83 (=5000/60) of these Searches are active on the platform at the same time. The effect of rate limiting will be that Searches will return no results until the hour interval is over.

• Rate limiting kicks in even earlier for Searches that return a lot of hits, typically more than 25 hits/minute. Each page of 25 results after the first one requires a new request and thus effectively counts as a new Search.

• In reality, the effective maximum throughput strongly depends on how efficient the pages are filled. If all of them are full the theoretical maximum throughput is 35 Posts/sec. If each of them just contains a single post, then the effective throughput is only 1.4 Posts/sec. In reality, the Instagram API rarely uses all available bandwidth, so +/- 3-5 Posts/sec is a more typical number often observes as a maximum for this API.

Graph API Limits (Page and Post Searches)

The integration of the Facebook Graph API into Viz Social forms the basis of the Facebook Page and Post Searches and is described in Facebook. Known volume or performance restrictions are:

• Facebook limits API access when more than 600 searches/calls per interval of ten minutes are made. With the default Viz Social set-up where Searches are repeated once a minute, this implies that the interface will be rate-limited by Facebook when more than 60 (=600/10) Facebook Page, Term & Post Searches instances are active at the same time. The effect of rate limiting will be that Searches will return no results until the 10-minute interval is over.

• Rate limiting kicks in even earlier for Searches that return a lot of hits, typically more than ten hits/minute. Each page of ten results after the first one effectively counts as a new Search.

• When a Facebook Page Search also monitors the comments of N posts, this counts as N Searches.

• In reality the effective maximum throughput strongly depends on how efficient the pages are filled. If all of them are full the theoretical maximum throughput is 10 Posts/sec. If each of them just contains a single post, then the effective throughput is only 1.0 Posts/sec. In reality the Instagram API rarely uses all available bandwidth, so +/- 3 Posts/sec is a more typical number often observes as a maximum for this API.