HeartBeats Failed and reconnection

Surya 1

Hello @admin and fellow traders,

Im using Python SDK latest version 1.2.2

Today I faced a situation, which needs attention. Im running in two different strategies in two accounts.

Issue 1 :
In one account I believe there was a network blip which caused HeartBeats Failed message. Since there is a reconnect module for this purpose it was giving message its trying to reconnect

Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 2 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 2 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 2 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 2 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 2 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 2 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 2 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 2 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 2 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 2 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 2 seconds
HeartBeats Failed
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 2 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 2 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds
Retrying connection. Retry attempt count: 1. Next retry in around: 3 seconds

and this went on till end of the session. Log file had 11453 lines of such Retry message and 15721 HearBeat Failed message. But I was still getting ticks. I believe SDK is not realizing that reconnection is established. Continuous reconnection tries gave socket error and other issues.

Issue: 2

In my other account it have HeartBeat Failed message but no Retrying connection message. I have subscribed to three scrips BankNifty spot and 2x CE/PE. Bank nifty spot price was frozen but CE/PE ticks continue to come.

Kindly have a look at this issues. I believe this happened due to blip in the network, but this is not far from real situations. Thanks.

admin

Hi @rajanprabu we have noticed your issue we will revert yo back Asap.

admin

Hi @rajanprabu Its is working fine from our end. wb code.PNG

jiwan1986

This post is deleted!

jiwan1986

@admin I'm also facing the same issue. I'm getting ticks but the log file has thousands of same reconnect messages.

admin

Hi @rjbanna we have noticed this issue and it has been reported to the team we will let you know once it is resolved.

Surya 1

@admin

@admin said in HeartBeats Failed and reconnection:

Its is working fine from our end.

It works fine when there is no network blips. I have requested reconnect module in python SDK exactly for this purpose. I faced this issue again today.

admin

Hi @rajanprabu said in HeartBeats Failed and reconnection:

It works fine when there is no network blips. I have requested reconnect module in python SDK exactly for this purpose. I faced this issue again today.
we will take your point into consideration on priority and revert you back.

punit

Hi @rajanprabu ,

What is the time stamp you faced this issue: I think I faced the same issue today.

HB Stopped after : 2021-03-23 11:51:29.992341 (11:52 Heart beat never arrived)
tick stopped at : 2021-03-23 11:56:18.1555
Script ended at : 2021-03-23 12:09:29

This needed to fixed as SL/PB depends upon the ticks and we cannot afford ticks breaking for 10 minutes.

BR,
Ashok

punit

@admin ,
Also I could see below exception in logs on another ws script:

{'ak': 'ok', 'msg': 'heartbeat', 'task': 'hb', 'ts': '2021-03-23 10:52:06.848540'}
Unhandled Error
Traceback (most recent call last):
  File "/home/trader/.local/lib/python3.8/site-packages/twisted/python/log.py", line 103, in callWithLogger
    return callWithContext({"system": lp}, func, *args, **kw)
  File "/home/trader/.local/lib/python3.8/site-packages/twisted/python/log.py", line 86, in callWithContext
    return context.call({ILogContext: newCtx}, func, *args, **kw)
  File "/home/trader/.local/lib/python3.8/site-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/home/trader/.local/lib/python3.8/site-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
--- <exception caught here> ---
  File "/home/trader/.local/lib/python3.8/site-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite
    why = selectable.doRead()
  File "/home/trader/.local/lib/python3.8/site-packages/twisted/internet/tcp.py", line 243, in doRead
    return self._dataReceived(data)
  File "/home/trader/.local/lib/python3.8/site-packages/twisted/internet/tcp.py", line 249, in _dataReceived
    rval = self.protocol.dataReceived(data)
  File "/home/trader/.local/lib/python3.8/site-packages/twisted/protocols/tls.py", line 330, in dataReceived
    self._flushReceiveBIO()
  File "/home/trader/.local/lib/python3.8/site-packages/twisted/protocols/tls.py", line 300, in _flushReceiveBIO
    self._flushSendBIO()
  File "/home/trader/.local/lib/python3.8/site-packages/twisted/protocols/tls.py", line 252, in _flushSendBIO
    bytes = self._tlsConnection.bio_read(2 ** 15)
  File "/home/trader/.local/lib/python3.8/site-packages/OpenSSL/SSL.py", line 1782, in bio_read
    self._handle_bio_errors(self._from_ssl, result)
  File "/home/trader/.local/lib/python3.8/site-packages/OpenSSL/SSL.py", line 1757, in _handle_bio_errors
    raise ValueError("unknown bio failure")
builtins.ValueError: unknown bio failure


# Job completed successfully at 2021/03/23 10:53:06 (GMT+5.5).
# End of log.

Surya 1

@Ashok

@Ashok said in HeartBeats Failed and reconnection:

builtins.ValueError: unknown bio failure

its a twisted/autobahn package error. which SDK version are you using ?

punit

@rajanprabu , my sdk is 1.1.0

Surya 1

@Ashok Please update your SDK to 1.2.2. That will fix your problems.. ( I hope )

admin

Hi@rajanprabu we have tested the we socket today it is working fine from our side.