Web socket scripts stop at random times


  • @rajanprabu I got it one time today.


  • @rjbanna @rajanprabu , Yes I got it many times today.
    I do not suspect network here. Zerodha ws is working on same set-up without any stoppage or issues.

    {'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:24:40.171811'}
    {'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:24:40.173677'}
    {'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:24:40.173925'}
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats FailedHeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats FailedHeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    {'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:24:40.325233'}
    {'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:24:40.326658'}
    [2021/03/31 14:24:40] HeartBeats Failed
    {'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:24:40.329653'}
    {'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:24:40.330938'}
    {'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:24:40.332719'}
    {'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:24:40.334177'}
    {'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:24:40.360006'}
    {'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:24:40.361450'}
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats FailedHeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats FailedHeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats FailedHeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats FailedHeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats FailedHeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:40] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats FailedHeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    {'name': 'tm', 'tvalue': '31/03/2021 14:24:41'}
    {'name': 'tm', 'tvalue': '31/03/2021 14:24:41'}
    {'name': 'tm', 'tvalue': '31/03/2021 14:24:41'}
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed
    [2021/03/31 14:24:41] HeartBeats Failed


  • Hi @Ashok we are running web socket from last 2 hours we will update you on the issue.


  • @Ashok

    what I meant was whenever this happened I stopped the script and restarted it. this is what I meant by happened three times..

    were you still getting the data even when it was printing HeartBeats Failed ??


  • @rajanprabu I was getting the data even when heartbeat failed. So I think there is issue with the heartbeat logic.


  • @rajanprabu Looks like during print of HB failed, no data, but once it reconnects, data starts to flow.
    but there is a pause between hb failed and reconnect. also I could see lot of reconnects happening.
    In below snippet, all those logs with userid are reconnect I suppose.

    HeartBeats Failed
    HeartBeats Failed
    HeartBeats Failed
    <function on_connect at 0x7f5953218550>
    {'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
    HeartBeats Failed
    <function on_connect at 0x7f5953218550>
    {'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
    {'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:55:33.387510'}
    {'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.387881'}
    <function on_connect at 0x7f5953218550>
    {'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
    {'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:55:33.394461'}
    {'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.394752'}
    {'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.394985'}
    {'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.395571'}
    <function on_connect at 0x7f5953218550>
    {'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
    {'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:55:33.397835'}
    {'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.398078'}
    {'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:55:33.401344'}
    {'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:55:33.402240'}
    {'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.402513'}
    {'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.402945'}
    HeartBeats Failed
    HeartBeats Failed
    HeartBeats Failed
    HeartBeats Failed
    HeartBeats Failed
    HeartBeats Failed
    {'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:55:33.467947'}
    {'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:55:33.468794'}
    HeartBeats Failed
    HeartBeats Failed
    HeartBeats Failed
    HeartBeats Failed
    HeartBeats Failed
    HeartBeats Failed
    <function on_connect at 0x7f5953218550>
    {'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
    <function on_connect at 0x7f5953218550>
    {'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
    <function on_connect at 0x7f5953218550>
    {'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
    <function on_connect at 0x7f5953218550>
    {'task': 'cn', 'channel': '', 'token': '0968362079', 'user': 'REHR1010', 'acctid': 'REHR1010'}
    {'ak': 'ok', 'msg': 'connected', 'task': 'cn', 'ts': '2021-03-31 14:55:33.507141'}
    {'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-31 14:5


  • Hello @Ashok @rajanprabu @rjbanna web socket worked completely fine from our end.


  • @admin Try to run whole day. You might get the same issue.


  • Hi @rjbanna we will try to review the logs and update you on this issue.Kindly post your client code @ashok to analyze the logs.


  • @admin below is my websocket file. insertQWatcher,get* functions are custom ones.

    ## WebSocket
    from logging import exception
    import logging
    from smartapi import WebSocket
    import greenstalk
    import json
    from datetime import datetime
    from ab.getIdxList import getIdxList
    from ab.logConfig import getLogger
    from ab.insertQWatcher import insertQWatcher
    
    logger      = getLogger('scr_ws_orb')
    atFile      = open('/opt/kite/ab/token/abToken.fi','r')
    atData      = json.loads(atFile.read())
    bnftyQ      = greenstalk.Client(('127.0.0.1', 11300),use = 'bnftyQ', watch = 'bnftyQ')
    qList       = 'bnftyQ'   ## Important to add the queue list
    
    FEED_TOKEN  = atData['feedToken']
    CLIENT_CODE = 'REHR1010'
    nextWeek    = True
    ##This function will return token in required format : 'nse_fo|58999&nse_fo|59000'
    token       = getIdxList('BANKNIFTY',10,nextWeek)  
    task        = 'mw' #"mw"|"sfi"|"dp"
    ss          = WebSocket(FEED_TOKEN, CLIENT_CODE)
    
    logger.info('Adding tokens into queue watcher!!!')
    insertQWatcher(token,qList)
    logger.info('Added tokens into queue watcher!!!')
    
    def on_tick(ws, tick):
    
        try:
            for i in tick:
                if 'task' in i.keys():
                    ts          = datetime.now()
                    i['ts']     = ts.strftime('%Y-%m-%d %H:%M:%S.%f')
                    tickJson    = json.dumps(i)
                    junkQ.put(tickJson)
                    print(i)
                    continue
    
                elif i['name'] == 'sf' and 'ltp' in i.keys():   
                    ts          = datetime.now()
                    i['ts']     = ts.strftime('%Y-%m-%d %H:%M:%S.%f')
                    i['scr']    = 'EQ'
                    tickJson    = json.dumps(i)
                    bnftyQ.put(tickJson)        ##Add tick data to queue
    
                else:
                    print(i)
                    continue
    
        except Exception as e:
            print(format(e))
    
    def on_connect(ws, response):
        ws.websocket_connection() # Websocket connection  
        ws.send_request(token,task)
    
    def on_close(ws, code, reason):   
        ##When ws.stop() is enabled, the script will get stopped when no tick/hb fail
        #ws.stop()
        pass
    
    # Assign the callbacks.
    ss.on_ticks = on_tick
    ss.on_connect = on_connect
    ss.on_close = on_close
    
    ss.connect()
    

  • Hi we have tested web socket from today it is working fine from our end.


  • @admin I'm also facing this issue, WebSocket connection closes at random times. around 11AM or around 1:30 PM or around 2:30 PM. sometimes it stops multiple times a day. It happens almost daily.


  • Hi @pavank we will check it from our end and update you on this.