c# - Listing Friends — Hitting auth failure mid-process

So, I'm trying to the friends of all of someone's followers. Not recursively digging, just that single level.

I'm able to pull the followers just fine (and it looks like it can handle more than 75k followers, so I'm 90% sure I have the rate limits fine there).

But my code to pull the friends is throwing authentication errors after it returns 15 of the followers friends.

In other words, in this bit of code:

for (var i = 0; i < followerList.Count; i++)
{
await followersFriends(followerList[i]);
}


I'm getting this error when i == 15:

i == 15

LinqToTwitter.TwitterQueryException
HResult=0x80131500
Message={"request":"/1.1/friends/ids.json","error":"Not authorized."} - Please visit the LINQ to Twitter FAQ (at the HelpLink) for help on resolving this error.

It does appear to completely get 15 lists, it's the 16th that throws it. I have the feeling that this is not a coincidence that that's the same as the rate limit, albeit an odd error if it is a rate limit.

I'll throw a check to see if we're on a multiple of 15 and do a 15 minute pause, but, on the chance that that doesn't work, does anyone have a theory as to what else could be going on here?

ETA: I did try redoing my authentication (Doing application based) on each run through followersFriends. Didn't help.

followersFriends

ETA2:

This is the new calling code:

for (var i = 0; i < followerList.Count; i++)
{
if (i % 15 == 0 && i != 0) {
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - i = " + i + " and user = " + followerList[i]);
PauseThatRefreshes("friends");
RunMe();
}

await followersFriends(followerList[i]);
}


And here's the code being called:

private async Task followersFriends(ulong uid)
{
var twitterCtx2 = new TwitterContext(m_foo);

Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - *** Start Friend Get for " + uid.ToString() + " ***");

long cursor = -1;

do
{
var friendList = await (from friend in twitterCtx2.Friendship where friend.Type == FriendshipType.FriendIDs && friend.UserID == uid.ToString() && friend.Cursor == cursor select friend).SingleOrDefaultAsync();

if (twitterCtx2.RateLimitRemaining <= 2)
{
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - *** Pausing A ***");
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - Friends Rate Limit Remaining at Pause A: " +
twitterCtx2.RateLimitRemaining.ToString());
PauseThatRefreshes("friends");
}

if (friendList != null &&
friendList.IDInfo != null &&
friendList.IDInfo.IDs != null) {
cursor = friendList.CursorMovement.Next;
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - Friends Rate Limit Remaining: " + twitterCtx2.RateLimitRemaining.ToString());
friendList.IDInfo.IDs.ForEach(id => Output(uid.ToString(), id.ToString()));
}

if (twitterCtx2.RateLimitRemaining <= 2)
{
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - *** Pause B ***");
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - Friends Rate Limit Remaining at Pause B: " + twitterCtx2.RateLimitRemaining.ToString());
PauseThatRefreshes("friends");
}
} while (cursor != 0);

Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - *** Stop Friend Get for " + uid.ToString() + " ***");
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - Friends Rate Limit Remaining at stop: " + twitterCtx2.RateLimitRemaining.ToString());

return;
}


ETA just for clarification, the code that is calling the followersFriends. Yes, I know, I didn't change my button name =D This is more about collecting the data and I prefer a form as opposed to a command line... I've no idea why. Anyways, neither here nor there:

private async void button2_Click(object sender, EventArgs e)
{

var twitterCtx = new TwitterContext(m_foo);
string header = { "Account,Follows" };
File.WriteAllLines(@"C:tempfollowersFriends.txt", header);

//List to store ids of followers of main account
List<ulong> followerList = new List<ulong>();
// get main accounts followers and put into an array
long cursor = -1;
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - *** Start Follower Get ***");
do
{

var followers =
await
(from follower in twitterCtx.Friendship
where follower.Type == FriendshipType.FollowerIDs &&
follower.ScreenName == "[[redacted]]" &&
follower.Cursor == cursor
select follower)
.SingleOrDefaultAsync();
if (followers != null &&
followers.IDInfo != null &&
followers.IDInfo.IDs != null
&&
followers.CursorMovement != null)
{
cursor = followers.CursorMovement.Next;
followers.IDInfo.IDs.ForEach(id =>
followerList.Add(id));
}

if (twitterCtx.RateLimitRemaining <= 2)
PauseThatRefreshes("followers");
} while (cursor != 0);
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - *** Done Follower Get ***");

for (var i = 0; i < followerList.Count; i++)
{
if (i % 15 == 0 && i != 0)
{
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - i = " + i
+ " and user = " + followerList[i]);
PauseThatRefreshes("friends");
RunMe();
}
await followersFriends(followerList[i]);
}
}


followerList

1 Answer
1

If I'm interpreting this correctly, you're trying to get a list of a user's follower's friends. Notice how you're nesting the FriendIDs query inside of the FollowerIDs query. Twitter is not allowing this pattern for reasons I'm not aware of.

FriendIDs

FollowerIDs

That said, the good news is that there is a work-around where you can query all followers first and then query friends. Here's an example:

var allFollowerIds = new List<ulong>();
Friendship followers;
long cursor = -1;
do
{
try
{
followers =
await
(from follower in twitterCtx.Friendship
where follower.Type == FriendshipType.FollowerIDs &&
follower.UserID == "15411837" &&
follower.Cursor == cursor &&
follower.Count == 200
select follower)
.SingleOrDefaultAsync();
}
catch (TwitterQueryException tqe)
{
Console.WriteLine(tqe.ToString());
break;
}

if (followers != null &&
followers.IDInfo != null &&
followers.IDInfo.IDs != null)
{
cursor = followers.CursorMovement.Next;

allFollowerIds.AddRange(followers.IDInfo.IDs);
}

} while (cursor != 0);

cursor = -1;
foreach (var uid in allFollowerIds)
{
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - *** Start Friend Get for " + uid.ToString() + " ***");

do
{
var friendList = await (from friend in twitterCtx.Friendship where friend.Type == FriendshipType.FriendIDs && friend.UserID == uid.ToString() && friend.Cursor == cursor select friend).SingleOrDefaultAsync();

if (twitterCtx.RateLimitRemaining <= 2)
{
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - *** Pausing A ***");
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - Friends Rate Limit Remaining at Pause A: " +
twitterCtx.RateLimitRemaining.ToString());
//PauseThatRefreshes("friends");
}

if (friendList != null &&
friendList.IDInfo != null &&
friendList.IDInfo.IDs != null)
{
cursor = friendList.CursorMovement.Next;
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - Friends Rate Limit Remaining: " + twitterCtx.RateLimitRemaining.ToString());
//friendList.IDInfo.IDs.ForEach(id => Output(uid.ToString(), id.ToString()));
}

if (twitterCtx.RateLimitRemaining <= 2)
{
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - *** Pause B ***");
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - Friends Rate Limit Remaining at Pause B: " + twitterCtx.RateLimitRemaining.ToString());
//PauseThatRefreshes("friends");
}
} while (cursor != 0);

Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - *** Stop Friend Get for " + uid.ToString() + " ***");
Console.WriteLine(DateTime.Now.ToString("h:mm:ss tt") + " - Friends Rate Limit Remaining at stop: " + twitterCtx.RateLimitRemaining.ToString());
}


Notice the allFollowerIds - it's an accumulator for all of the follower IDs, populated for each cursor iteration. After you've obtained all of those IDs, then get their friends, which you can see this example does via a foreach loop.

allFollowerIds

foreach

Another tip - notice how I set the Count property on the FollowerIDs query to 200. The default is 20, which means that not setting count increases the probability of encountering rate limits because you have to make more queries. The max is 200, which you should set Count to for improving performance and preserving rate limit.

Count

FollowerIDs

200

20

200

Count

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.