Unpredictable Invalid Token response from historical API

pranav

I am trying to get data from historical API and it unpredictably gives invalid token error in response. But if I try to run the same script 2-3 times, it gives response.
2-3 times it fails and next time gives response.

Here is my code snippet:

from smartapi import WebSocket 
from smartapi import SmartConnect
from config import authInfo # file for storing api key, etc
import json
import requests as req

smartApi= SmartConnect(api_key= authInfo['apiKey'])

login= smartApi.generateSession(authInfo['clientCode'], authInfo['password'])

refreshToken= login['data']['refreshToken']

feedToken= smartApi.getfeedToken()

profile= smartApi.getProfile(refreshToken)

print('login jwt token: ', login['data']['jwtToken'])
print('auth may be: ', smartApi.Authorization)
t= smartApi.generateToken(refreshToken)

print("success")

url= 'https://apiconnect.angelbroking.com/rest/secure/angelbroking/historical/v1/getCandleData'

headers= {
    'X-PrivateKey': authInfo['apiKey'], 
    'Accept': 'application/json', 
    'X-SourceID': 'WEB', 
    'X-ClientLocalIP': 'CLIENT_LOCAL_IP',
    'X-ClientPublicIP': 'CLIENT_PUBLIC_IP',
    'X-MACAddress': 'MAC_ADDRESS',
    'X-UserType': 'USER', 
    'Authorization': login['data']['jwtToken'], 
    'Content-Type': 'application/json'
}

# token 3721 for TATA Comm
reqObj= {
    'exchange': 'NSE',
    'symboltoken': '3721', 
    'interval': 'ONE_DAY',
    'fromdate': '2021-03-01 09:15',
    'todate': '2021-03-30 15:30'
}

r= req.post(url, data= json.dumps(reqObj), headers= headers)
print('successfully connected to historical api')
# print('encoding: ', r.encoding)
data= r.text
print('data: \n', data)

And this is the response I get for r.text :

{"success":false,"message":"Invalid Token","errorCode":"AG8001","data":""}

It comes 2-3 times.

Then finally, this comes
{"status":true,"message":"SUCCESS","errorcode":"","data":"2021-03-02T00:00:00+05:30,1120.80,1333.95,1099.20,1202.95,1147120\n2021-03-03T00:00:00+05:30,1260.00,1342.15,1220.10,1228.75,887076\n2021-03-04T00:00:00+05:30,1221.00,1259.90,1220.00,1242.30,321693\n2021-03-05T00:00:00+05:30,1260.00,1278.30,1236.80,1251.00,443257\n2021-03-08T00:00:00+05:30,1251.15,1367.65,1251.00,1333.15,1021407\n2021-03-09T00:00:00+05:30,1343.80,1350.00,1264.25,1300.05,343657\n2021-03-10T00:00:00+05:30,1305.15,1343.85,1280.00,1290.35,286591\n2021-03-12T00:00:00+05:30,1300.00,1334.00,1267.35,1291.75,182248\n2021-03-15T00:00:00+05:30,1300.00,1330.00,1265.00,1293.80,421227\n2021-03-16T00:00:00+05:30,1210.00,1252.90,1201.25,1214.80,8983613\n2021-03-17T00:00:00+05:30,1177.00,1189.70,1135.10,1141.10,5902258\n2021-03-18T00:00:00+05:30,1162.90,1173.70,1142.10,1166.20,1555385\n2021-03-19T00:00:00+05:30,1153.00,1180.00,1134.05,1175.20,1229072\n2021-03-22T00:00:00+05:30,1180.00,1194.70,1140.35,1160.65,1953673\n2021-03-23T00:00:00+05:30,1149.00,1151.70,1107.40,1113.20,1037779\n2021-03-24T00:00:00+05:30,1112.90,1120.00,1060.00,1096.45,1549436\n2021-03-25T00:00:00+05:30,1094.95,1094.95,1052.00,1072.35,927086\n2021-03-26T00:00:00+05:30,1087.40,1133.70,1073.10,1117.85,908790\n2021-03-30T00:00:00+05:30,1133.00,1137.90,1086.20,1093.00,390677"}

Please tell me if I'm doing something wrong.
PS: 'X-ClientLocalIP': 'CLIENT_LOCAL_IP',
'X-ClientPublicIP': 'CLIENT_PUBLIC_IP',
'X-MACAddress': 'MAC_ADDRESS'

I'm passing these as it is, ie. not the actual value but the way it written in the snippet. Hopefully that should not be a problem (OR is it??)

Also if someone can explain, how to interpret the json response as it is getting really tough to understand the response

admin

Hi @pranav we will check and revert you back.

admin

Hi @pranav As we tried to hit historical data API multiple times We are able to easily get the data. Please try to restart the kernel and try to run the code again.

pranav

@admin Still having the same issue. I'll try in a cloud server now. May be, it will work fine there. But what about the structure of data I'm getting in response. It's hard to understand. Can you share a code* snippet for converting it to some useful format (like pandas data frame or json with key value pairs) as right now I'm unable to figure out what value is what. Eg. in
2021-03-02T00:00:00+05:30,1120.80,1333.95,1099.20,1202.95,1147120\n
it would be better if it was mentioned like
{
time_modified: '02T00:00:00+05:30',
open: 1120.80,
high: 1333.95,
...
}

sorry if I'm asking for too much

jiwan1986

@pranav Currently the response is in a string format. They are working on to return JSON data. So it would be easy to parse.

pranav

@rjbanna All right , when can I expect it to be done? I actually need it for a project. if I somehow parse the current format but it stops working in future.

@admin can you give a tentative time period?

jiwan1986

@pranav I don't have any timeline. You can ask @admin

admin

Hi @pranav we can expect it to be ready after 3 to 4 working days.

pranav

@admin great

Surya 1

@pranav

You dont have to wait.. you can do it yourself until team implements it.. use the snippet below.. you will get a data frame.. there must be other easier ways to do it.. I just write it quickly for you. If you want json you convert it before df or simply df.to_json()

import pandas as pd

params = { 
"exchange": "NSE", 
"symboltoken": "3045", 
"interval": "ONE_MINUTE", 
 "fromdate": "2021-02-08 09:15", 
 "todate": "2021-02-08 09:45" 
}                                                                                                                                                                                    
resp =  angel_api.getCandleData(params) 

df = pd.DataFrame(resp['data'].split('\n'), columns=["data"]) 
x =  [x.split(',') for x in df['data'].values.tolist()] 
columns = ['Timestamp', 'open', 'high', 'low', 'close', 'volume']
df  = pd.DataFrame(x, columns= columns)

Hope this helps.

pranav

@admin any updates on this?

admin

Hi @pranav we will update you on this.