Web socket scripts stop at random times

Surya 1

@Ashok @rjbanna

with new SDK on MCX ?

jiwan1986

@rajanprabu Yes

punit

@admin , Any update on this? Facing exact same issues still. ws is not reliable at all.

Surya 1

@Ashok

Im running from morning and I dint face any issue. @rjbanna what about you ?

admin

Hi @Ashok can you post the snip here.

punit

@admin

{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-30 13:59:11.170880'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-30 13:59:11.171432'}
{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-30 13:59:11.171963'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-30 13:59:11.172687'}
[2021/03/30 13:59:11] HeartBeats Failed
[2021/03/30 13:59:11] HeartBeats Failed
[2021/03/30 13:59:12] HeartBeats Failed
{'name': 'tm', 'tvalue': '30/03/2021 13:59:12'}
[2021/03/30 13:59:12] HeartBeats Failed
{'name': 'tm', 'tvalue': '30/03/2021 13:59:12'}
{'name': 'tm', 'tvalue': '30/03/2021 13:59:12'}
[2021/03/30 13:59:12] HeartBeats Failed
[2021/03/30 13:59:12] HeartBeats Failed
[2021/03/30 13:59:13] HeartBeats Failed
{'name': 'tm', 'tvalue': '30/03/2021 13:59:13'}
[2021/03/30 13:59:13] HeartBeats Failed
[2021/03/30 13:59:13] HeartBeats Failed
{'name': 'tm', 'tvalue': '30/03/2021 13:59:13'}
{'name': 'tm', 'tvalue': '30/03/2021 13:59:13'}
[2021/03/30 13:59:13] <function on_connect at 0x7ff5b750d5e0>
{'task': 'cn', 'channel': '', 'token': '0921225492', 'user': 'REHR1010', 'acctid': 'REHR1010'}
{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-30 13:59:13.286596'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-30 13:59:13.287143'}
[2021/03/30 13:59:13] HeartBeats Failed
[2021/03/30 13:59:13] HeartBeats Failed
[2021/03/30 13:59:13] HeartBeats Failed
[2021/03/30 13:59:14] HeartBeats Failed
{'name': 'tm', 'tvalue': '30/03/2021 13:59:14'}
{'name': 'tm', 'tvalue': '30/03/2021 13:59:14'}
[2021/03/30 13:59:14] HeartBeats Failed
{'name': 'tm', 'tvalue': '30/03/2021 13:59:14'}
[2021/03/30 13:59:14] HeartBeats Failed
[2021/03/30 13:59:14] HeartBeats Failed
[2021/03/30 13:59:14] HeartBeats Failed
[2021/03/30 13:59:14] HeartBeats Failed
[2021/03/30 13:59:14] <function on_connect at 0x7ff5b750d5e0>
{'task': 'cn', 'channel': '', 'token': '0921225492', 'user': 'REHR1010', 'acctid': 'REHR1010'}
{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-30 13:59:15.025724'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-30 13:59:15.026091'}
{'name': 'tm', 'tvalue': '30/03/2021 13:59:15'}
{'name': 'tm', 'tvalue': '30/03/2021 13:59:15'}
[2021/03/30 13:59:15] HeartBeats Failed
{'name': 'tm', 'tvalue': '30/03/2021 13:59:15'}
[2021/03/30 13:59:15] HeartBeats Failed
[2021/03/30 13:59:15] HeartBeats Failed
[2021/03/30 13:59:15] HeartBeats Failed
[2021/03/30 13:59:15] HeartBeats Failed
[2021/03/30 13:59:15] HeartBeats Failed
{'name': 'tm', 'tvalue': '30/03/2021 13:59:16'}
{'name': 'tm', 'tvalue': '30/03/2021 13:59:16'}
{'name': 'tm', 'tvalue': '30/03/2021 13:59:16'}
[2021/03/30 13:59:16] HeartBeats Failed
[2021/03/30 13:59:16] HeartBeats Failed
[2021/03/30 13:59:16] HeartBeats Failed
[2021/03/30 13:59:16] HeartBeats Failed
[2021/03/30 13:59:16] HeartBeats Failed
[2021/03/30 13:59:17] HeartBeats Failed
{'name': 'tm', 'tvalue': '30/03/2021 13:59:17'}
{'name': 'tm', 'tvalue': '30/03/2021 13:59:17'}
{'name': 'tm', 'tvalue': '30/03/2021 13:59:17'}
[2021/03/30 13:59:17] HeartBeats Failed
[2021/03/30 13:59:17] HeartBeats Failed
[2021/03/30 13:59:17] HeartBeats Failed
[2021/03/30 13:59:17] HeartBeats Failed
[2021/03/30 13:59:17] HeartBeats Failed
[2021/03/30 13:59:17] HeartBeats Failed
[2021/03/30 13:59:17] HeartBeats Failed
[2021/03/30 13:59:18] HeartBeats Failed

admin

Hi @Ashok we will update you on this.

jiwan1986

@rajanprabu @Ashok I also didn't face this issue today. It's working fine for me.

Surya 1

@rjbanna @Ashok

I got HeartBeat failed message three times today so far.. is it running well for you guys ? It may be my network

admin

Hi @rajanprabu we will test it from our end and revert you back.

jiwan1986

@rajanprabu I got it one time today.

punit

@rjbanna @rajanprabu , Yes I got it many times today.
I do not suspect network here. Zerodha ws is working on same set-up without any stoppage or issues.

{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:24:40.171811'}
{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:24:40.173677'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:24:40.173925'}
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats FailedHeartBeats Failed
[2021/03/31 14:24:40] HeartBeats FailedHeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:24:40.325233'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:24:40.326658'}
[2021/03/31 14:24:40] HeartBeats Failed
{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:24:40.329653'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:24:40.330938'}
{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:24:40.332719'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:24:40.334177'}
{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:24:40.360006'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:24:40.361450'}
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats FailedHeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats FailedHeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats FailedHeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats FailedHeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats FailedHeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:40] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats FailedHeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
{'name': 'tm', 'tvalue': '31/03/2021 14:24:41'}
{'name': 'tm', 'tvalue': '31/03/2021 14:24:41'}
{'name': 'tm', 'tvalue': '31/03/2021 14:24:41'}
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed
[2021/03/31 14:24:41] HeartBeats Failed

admin

Hi @Ashok we are running web socket from last 2 hours we will update you on the issue.

Surya 1

@Ashok

what I meant was whenever this happened I stopped the script and restarted it. this is what I meant by happened three times..

were you still getting the data even when it was printing HeartBeats Failed ??

jiwan1986

@rajanprabu I was getting the data even when heartbeat failed. So I think there is issue with the heartbeat logic.

punit

@rajanprabu Looks like during print of HB failed, no data, but once it reconnects, data starts to flow.
but there is a pause between hb failed and reconnect. also I could see lot of reconnects happening.
In below snippet, all those logs with userid are reconnect I suppose.

HeartBeats Failed
HeartBeats Failed
HeartBeats Failed
<function on_connect at 0x7f5953218550>
{'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
HeartBeats Failed
<function on_connect at 0x7f5953218550>
{'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:55:33.387510'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.387881'}
<function on_connect at 0x7f5953218550>
{'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:55:33.394461'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.394752'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.394985'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.395571'}
<function on_connect at 0x7f5953218550>
{'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:55:33.397835'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.398078'}
{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:55:33.401344'}
{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:55:33.402240'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.402513'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.402945'}
HeartBeats Failed
HeartBeats Failed
HeartBeats Failed
HeartBeats Failed
HeartBeats Failed
HeartBeats Failed
{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:55:33.467947'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.468794'}
HeartBeats Failed
HeartBeats Failed
HeartBeats Failed
HeartBeats Failed
HeartBeats Failed
HeartBeats Failed
<function on_connect at 0x7f5953218550>
{'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
<function on_connect at 0x7f5953218550>
{'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
<function on_connect at 0x7f5953218550>
{'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
<function on_connect at 0x7f5953218550>
{'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
{'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:55:33.507141'}
{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:5

admin

Hello @Ashok @rajanprabu @rjbanna web socket worked completely fine from our end.

jiwan1986

@admin Try to run whole day. You might get the same issue.

admin

Hi @rjbanna we will try to review the logs and update you on this issue.Kindly post your client code @ashok to analyze the logs.

punit

@admin below is my websocket file. insertQWatcher,get* functions are custom ones.

## WebSocket
from logging import exception
import logging
from smartapi import WebSocket
import greenstalk
import json
from datetime import datetime
from ab.getIdxList import getIdxList
from ab.logConfig import getLogger
from ab.insertQWatcher import insertQWatcher

logger      = getLogger('scr_ws_orb')
atFile      = open('/opt/kite/ab/token/abToken.fi','r')
atData      = json.loads(atFile.read())
bnftyQ      = greenstalk.Client(('127.0.0.1', 11300),use = 'bnftyQ', watch = 'bnftyQ')
qList       = 'bnftyQ'   ## Important to add the queue list

FEED_TOKEN  = atData['feedToken']
CLIENT_CODE = 'REHR1010'
nextWeek    = True
##This function will return token in required format : 'nse_fo|58999&nse_fo|59000'
token       = getIdxList('BANKNIFTY',10,nextWeek)  
task        = 'mw' #"mw"|"sfi"|"dp"
ss          = WebSocket(FEED_TOKEN, CLIENT_CODE)

logger.info('Adding tokens into queue watcher!!!')
insertQWatcher(token,qList)
logger.info('Added tokens into queue watcher!!!')

def on_tick(ws, tick):

    try:
        for i in tick:
            if 'task' in i.keys():
                ts          = datetime.now()
                i['ts']     = ts.strftime('%Y-%m-%d %H:%M:%S.%f')
                tickJson    = json.dumps(i)
                junkQ.put(tickJson)
                print(i)
                continue

            elif i['name'] == 'sf' and 'ltp' in i.keys():   
                ts          = datetime.now()
                i['ts']     = ts.strftime('%Y-%m-%d %H:%M:%S.%f')
                i['scr']    = 'EQ'
                tickJson    = json.dumps(i)
                bnftyQ.put(tickJson)        ##Add tick data to queue

            else:
                print(i)
                continue

    except Exception as e:
        print(format(e))

def on_connect(ws, response):
    ws.websocket_connection() # Websocket connection  
    ws.send_request(token,task)

def on_close(ws, code, reason):   
    ##When ws.stop() is enabled, the script will get stopped when no tick/hb fail
    #ws.stop()
    pass

# Assign the callbacks.
ss.on_ticks = on_tick
ss.on_connect = on_connect
ss.on_close = on_close

ss.connect()