[{"data":1,"prerenderedAt":65},["ShallowReactive",2],{"$fe8qa6kbg4po":3,"recommended":34},{"title":4,"path":5,"date":6,"updated":7,"tags":8,"excerpt":10,"readingMinutes":11,"commentCount":12,"html":13,"comments":14,"next":28,"prev":31},"Yahoo Pipes uses multiple IP's","\u002F2011\u002F12\u002Fyahoo-pipes-uses-multiple-ips\u002F","2011-12-15T20:17:00.004Z","2011-12-15T20:26:52.677Z",[9],"social graph analysis","For my Twitter analysis program I need access to quite a lot of data from the twitter api (about 200 requests per visitor). Unfortunately the twitter api has an awful rate limit of 150 requests per hour per IP, so it is…",2,3,"\u003Cp>For my Twitter analysis program I need access to quite a lot of data from the twitter api (about 200 requests per visitor). Unfortunately the twitter api has an awful rate limit of 150 requests per hour per IP, so it is impossible to route this all through my server. Fortunately the api accepts jsonp requests, so I decided to offload fetching the data to the client.\u003C\u002Fp>\n\u003Cp>The problem is that for most of my users 150 requests just won't cut it. After looking up a single person with 150 followers, the user has to wait for an hour. A caching solution is needed.\u003C\u002Fp>\n\u003Cp>I decided to create a workflow like this for every request that would normally go through the twitter api:\u003C\u002Fp>\n\u003Col>\n\u003Cli>Ask my cache at waleson.com for the info. If successful, return.\u003C\u002Fli>\n\u003Cli>Try the twitter api via json p. If successful, send to waleson.com cache before returning.\u003C\u002Fli>\n\u003Cli>Ask my server to act as a proxy to the twitter api (Actually I've hooked up some others webservers that I happen to manage that my server uses as proxies).\u003C\u002Fli>\n\u003C\u002Fol>\n\u003Cp>However, I had played with Yahoo Pipes some months ago. (And I _really_ liked it.) Pipes accepts jsonp requests too. Perhaps it could be used to bypass the 150\u002Fhour\u002Fip limit? Maybe not, maybe my Pipe is put on a single server and all requests will be made from that server's IP addresses?\u003C\u002Fp>\n\u003Cp>Well, I set up a pipe, made it do a request to a logger on my server and did about 60 results. As it turns out, Pipes uses multiple IPs (this contains 7 unique IP's, there might be many more, but this should do for now):\u003C\u002Fp>\n\u003Cp>\u003Ccode>   83.87.80.102   87.248.125.49   87.248.125.49   217.12.1.125   87.248.125.48   217.146.191.18   217.12.1.124   217.146.191.19   217.146.191.19   87.248.125.49   217.12.1.124   217.12.1.124   217.12.1.125   87.248.125.49   87.248.125.48   87.248.125.48   87.248.125.48   217.12.1.125   217.146.191.19   217.146.191.19   217.12.1.124   87.248.125.48   217.12.1.124   217.12.1.124   217.12.1.125   87.248.125.48   217.12.1.124   217.12.1.124   87.248.125.48   217.146.191.18   217.12.1.125   217.12.1.124   217.146.191.19   87.248.125.49   217.12.1.125   217.12.1.124   217.12.1.125   217.146.191.18\u003C\u002Fcode>\n\u003Ccode>217.12.1.125\u003C\u002Fcode>\n\u003Ccode>217.12.1.125\u003C\u002Fcode>\n\u003Ccode>87.248.125.49\u003C\u002Fcode>\n\u003Ccode>87.248.125.48\u003C\u002Fcode>\n\u003Ccode>217.146.191.19\u003C\u002Fcode>\n\u003Ccode>217.12.1.125\u003C\u002Fcode>\n\u003Ccode>217.146.191.18\u003C\u002Fcode>\n\u003Ccode>87.248.125.49\u003C\u002Fcode>\n\u003Ccode>87.248.125.49\u003C\u002Fcode>\n\u003Ccode>217.12.1.125\u003C\u002Fcode>\n\u003Ccode>87.248.125.49\u003C\u002Fcode>\n\u003Ccode>217.12.1.125\u003C\u002Fcode>\n\u003Ccode>217.12.1.125\u003C\u002Fcode>\n\u003Ccode>217.12.1.125\u003C\u002Fcode>\n\u003Ccode>217.12.1.124\u003C\u002Fcode>\n\u003Ccode>217.146.191.19\u003C\u002Fcode>\n\u003Ccode>217.12.1.124\u003C\u002Fcode>\n\u003Ccode>217.146.191.19\u003C\u002Fcode>\n\u003Ccode>217.146.191.19\u003C\u002Fcode>\n\u003Ccode>217.146.191.19\u003C\u002Fcode>\n\u003Ccode>217.12.1.124\u003C\u002Fcode>\n\u003Ccode>217.146.191.19\u003C\u002Fcode>\n\u003Ccode>217.12.1.124\u003C\u002Fcode>\n\u003Ccode>   \u003C\u002Fcode>\nUser agent: Mozilla\u002F5.0 (compatible; Yahoo Pipes 2.0; +http:\u002F\u002Fdeveloper.yahoo.com\u002Fyql\u002Fprovider) Gecko\u002F20090729 Firefox\u002F3.5.2\u003C\u002Fp>\n\u003Cp>So, I am currently putting Pipes somewhere in my workflow to act as another set of proxies to support even more users. Awesome!\u003C\u002Fp>\n",[15,20,24],{"author":16,"url":17,"date":18,"html":19},"Govt Jobs India","http:\u002F\u002Fwww.govtjobsforum.com","2012-02-15T08:07:17.119+01:00","Keep in mind pipes also have limitation.",{"author":21,"date":22,"html":23},"Jouke Waleson","2012-02-15T10:33:35.164+01:00","Thanks for the addition",{"author":25,"date":26,"html":27},"Anonymous","2015-09-18T12:53:23.535+02:00","very nice and good post.\u003Cbr \u002F>\u003Ca href=\"http:\u002F\u002Fwww.crackedsoftwaresdownload.com\u002Froot-explorer-pro-apk-app-android\u002F\" rel=\"nofollow\">Root explorer apk download android\u003C\u002Fa>",{"title":29,"path":30},"Moving to Google's App Engine","\u002F2011\u002F12\u002Fmoving-to-googles-app-engine\u002F",{"title":32,"path":33},"Coming soon, social graph analysis in your browser","\u002F2011\u002F12\u002Fcoming-soon-social-graph-analysis-in\u002F",[35,40,45,48,52,55,58,62],{"title":36,"path":37,"note":38,"series":39},"Mapping out Dark Code","\u002F2026\u002F03\u002Fmapping-out-dark-code\u002F","2026",false,{"title":41,"path":42,"note":43,"series":44},"The software that I love","\u002F2022\u002F10\u002Fthe-software-that-i-love\u002F","Series · 36 posts",true,{"title":46,"path":47,"note":38,"series":39},"Navigating the Fog of War: Age of Empires, AI Agents, and the Rise of Dark Code","\u002F2026\u002F05\u002Fnavigating-fog-of-war-age-of-empires-ai\u002F",{"title":49,"path":50,"note":51,"series":39},"The Future of Programming Systems - four thoughts","\u002F2024\u002F09\u002Fthe-future-of-programming-systems-four\u002F","2024",{"title":53,"path":54,"note":51,"series":39},"Always Optimize the Feedback Loop","\u002F2024\u002F07\u002Falways-optimize-feedback-loop\u002F",{"title":56,"path":57,"note":51,"series":39},"\"Security Is Our Top Priority\" is BS","\u002F2024\u002F07\u002Fsecurity-is-our-top-priority-is-bs\u002F",{"title":59,"path":60,"note":61,"series":39},"10x teams","\u002F2025\u002F02\u002F10x-teams\u002F","2025",{"title":63,"path":64,"note":61,"series":39},"Hiring, from the Other Side","\u002F2025\u002F10\u002Fhiring-from-other-side\u002F",1792876809202]