python - Extracting data in Tweepy in useable form (JSON) -

i using tweepy gather data on followers. able print returned data, no matter try, can't save data file in reuseable form.

below way got export txt file, when call file variable, sets character array , doesn't see useable array.

here tweepy call:

import sys import tweepy import json  # key info removed  auth = tweepy.oauthhandler(consumer_key, consumer_secret) auth.set_access_token(access_key, access_secret) api = tweepy.api(auth)  ids = [] page in tweepy.cursor(api.followers, screen_name="handle").pages():     ids.extend(page) print ids  open("followers.txt", 'w') f:     f.write(str(ids)) 

what file starts:

[user(follow_request_sent=false...  

if call data in

data = f.read() print data[0]  # returns '[' 

each page list of tweepy user objects. you're losing tweepy data structure when you: ids.extend(page).

try bit of code below:

with open("followers.txt", 'a') f:     #open file first     page in tweepy.cursor(api.followers, screen_name='handle').pages():         user_obj in page:   #iterate through each user object             json.dump(user_obj._json, f)  #dump each file, f             f.write("\n")  #you'll need martjin's answer below work. 

thanks martjin pieter's answer this question. can load , discretely access data. way modified code snippet jfile appened list called user_jsons (which equivalent data variable).

user_jsons = [] open("followers.txt", 'rb') f:     line in f:         while true:             try:                 jfile = json.loads(line)                 break             except valueerror:                 # not yet complete json value                 line += next(f)         user_jsons.append(jfile) 

now have list of json objects... [7] truncated

in [7]: user_jsons[0] out[7]: {u'blocked_by': false,           u'blocking': false,           u'contributors_enabled': false,           u'created_at': u'thu jan 30 18:33:13 +0000 2014',           ...  in [8]: user_jsons[0]['screen_name'] out[8]: u'some_users_handle' 

you may find ipython notebooks here useful resource, chapter 9 in particular.