Navigation

    SmartAPI Forum
    • Register
    • Login
    • Search
    • Categories
    • Popular
    • Groups
    • FAQs
    • API Docs

    Issue fetching Historical data

    Python SDK
    0
    12
    180
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • V
      Vypy1 last edited by

      I am facing a peculiar issue in fetching data. This wasn't a problem earlier when I fetched data using the same code. The code below loops through a few different date ranges to fetch data for that date range. I transform the fetched data into data frames of each date range and then concat it to return one single data frame.

      But when I see the excel sheet that I store the data in the dates that I have fetched the data for are arbitrary, it skips a lot of days in between, sometimes skips years. Starts not from the start date I have set and instead from some arbitrary date, I have given a snippet of the result below. Date starts from 2013 instead of requested 2011, it jumps from 2015 to 2018 and then 2019 without retrieving the complete data. Can anyone help here.

      def hist_data():
      
          dfs = []
          frm = ['2015-07-12 09:15', '2020-01-01 09:15']
          until = ['2020-12-31 15:30', '2022-11-17 15:30']
      
          for x, y in zip(frm, until):
      
              try:
                  historicParam = {
                      "exchange": "NSE",
                      "symboltoken": tokens,
                      "interval": 'ONE_DAY',
                      "fromdate": x,
                      "todate": y
                  }
                  dfs.append(pd.DataFrame(obj.getCandleData(historicParam)['data'],
                                          columns=['Date', 'Open', 'High', 'Low', 'Close', 'Volume']))
      
              except Exception as e:
                  print(f'Historic Api failed: {e}')
      
      
          df = pd.concat(x for x in dfs)
          df['Date'] = df['Date'].str.split('T').str[0]
      
          return df
      
      for tokens, symbols in zip(nifty_50['token'], nifty_50['symbol']):
      
          stock_data = hist_data()
          stock_data.to_csv(f'/Users/varadjoshi/Documents/Markets Data/Price Volume Data/Nifty 500 Stocks Data/{symbols}.csv', index=False)
          time.sleep(1)
          print(symbols)
      

      Output in the excel sheet below

      ![f9cd5502-b1d8-4f4d-9944-cffb397e39a3-image.png](/assets/uploads/files/1668693905626-f9cd5502-b1d8-4f4d-9944-cffb397e39a3-image.png) code_text
      
      ![dd9bcd9f-cbb0-4960-a0da-abac7e2a1955-image.png](/assets/uploads/files/1668693836453-dd9bcd9f-cbb0-4960-a0da-abac7e2a1955-image.png) code_text
      ```26/06/15	169.99	170.44	163.28	167.57	2150489
      29/06/15	166.51	166.84	160.28	165.81	1140046
      30/06/15	164.73	170.95	164.12	169.44	1197561
      01/07/15	168.08	170.29	166.03	166.81	792876
      02/07/15	164.42	174.04	164.42	172.46	1866024
      03/07/15	172.89	173.34	168.05	170.86	1611144
      06/07/15	168.05	173.46	166	172.8	2376410
      07/07/15	173.49	177.9	171.5	177.15	1906853
      08/07/15	175.64	175.91	167.57	168.87	3483626
      09/07/15	168.96	169.26	166.24	166.96	1521201
      10/07/15	167.66	169.2	165.33	167.36	655030
      26/12/18	283.81	289.77	277.8	286.35	3413076
      27/12/18	288.95	292.58	277.65	280.03	5553470
      28/12/18	280.55	289.8	280.4	287.83	1932203
      31/12/18	291.04	291.25	285.38	286.32	1383559
      01/01/19	288.23	288.35	282.79	285.2	986138
      02/01/19	285.08	286.77	278.92	281.76	1550368
      03/01/19	281.15	286.41	274.44	277.04	1653258
      04/01/19	277.95	283.99	276.38	280.61	1745111
      07/01/19	284.42	285.29	278.67	282.15	1675020
      08/01/19	279.34	282.3	275.95	279.16	1713143
      09/01/19	280.49	281.7	273.2	277.77	1790489
      10/01/19	277.77	278.95	274.87	277.77	1632163
      11/01/19	277.1	278.16	270.27	273.87	2006360
      14/01/19	270.21	273.23	262.96	265.44	3997561
      15/01/19	265.98	269.91	265.04	268.43	3103563
      16/01/19	267.52	272.51	266.8	270.15	2311584
      17/01/19	271.96	276.35	268.07	270.66	2779574
      18/01/19	272.33	273.14	265.13	266.92	1582590
      V A 2 Replies Last reply Reply Quote 1
      • V
        Vypy1 @Vypy1 last edited by

        @Vypy1 @admin any update on this?

        V 6 Replies Last reply Reply Quote 0
        • V
          Vypy1 @Vypy1 last edited by

          @Vypy1 I did some digging and it seems the request itself is not receiving the correct data.

          I used the below code to check if your _request function is receiving the right data, but there seems to be some problem here.

          My date requested from is 2019-1-1 and I start getting data from 2020-1-1

          params = historicParam = {
                      "exchange": "NSE",
                      "symboltoken": 11821,
                      "interval": 'ONE_DAY',
                      "fromdate": '2019-01-01 09:15',
                      "todate": '2022-11-17 15:30'
                  }
          route = 'api.candle.data'
          method = 'POST'
          
          print(obj._request(route=route, method=method, parameters=params)['data'])
          
          [['2020-11-10T00:00:00+05:30', 104.5, 105.4, 101.15, 101.8, 662663], ['2020-11-11T00:00:00+05:30', 102.55, 103.0, 100.5, 101.85, 653453], ['2020-11-12T00:00:00+05:30', 101.9, 103.2, 101.0, 102.0, 594231], ['2020-11-13T00:00:00+05:30', 102.0, 103.9, 100.0, 101.45, 1197144], ['2020-11-17T00:00:00+05:30', 102.2, 110.85, 101.5, 107.0, 2480276], ['2020-11-18T00:00:00+05:30', 107.0, 110.9, 106.75, 109.45, 2004444], ['2020-11-19T00:00:00+05:30', 109.0, 112.0, 105.7, 107.9, 1102268], ['2020-11-20T00:00:00+05:30', 108.75, 114.0, 108.2, 112.7, 2950348], ['2020-11-23T00:00:00+05:30', 113.8, 113.8, 111.1, 111.6, 1840552], ['2020-11-24T00:00:00+05:30', 111.95, 113.6, 111.0, 111.4, 2382269], ['2020-11-25T00:00:00+05:30', 112.25, 116.0, 111.3, 115.2, 3936048], ['2020-11-26T00:00:00+05:30', 116.05, 118.0, 114.05, 115.65, 1452319], ['2020-11-27T00:00:00+05:30', 116.0, 120.0, 114.95, 118.1, 2300417], ['2020-12-01T00:00:00+05:30', 118.2, 127.35, 116.55, 125.65, 3557627]
          
          1 Reply Last reply Reply Quote 0
          • V
            Vypy1 @Vypy1 last edited by

            @Vypy1 @admin any update here?

            V 1 Reply Last reply Reply Quote 0
            • V
              Vypy1 @Vypy1 last edited by

              @Vypy1 @admin why is there no response on this? You make the API, if your users have a question its better to respond to them than be completely silent about it.

              1 Reply Last reply Reply Quote 0
              • V
                Vypy1 @Vypy1 last edited by

                @Vypy1 @admin what is the update on this?

                1 Reply Last reply Reply Quote 0
                • V
                  Vypy1 @Vypy1 last edited by

                  @Vypy1 @admin waiting for your response

                  1 Reply Last reply Reply Quote 0
                  • V
                    Vypy1 @Vypy1 last edited by

                    @Vypy1 @admin I have been following up for a response on this and you guys don't care to even respond. What is the issue?

                    1 Reply Last reply Reply Quote 0
                    • V
                      Vypy1 @Vypy1 last edited by

                      @Vypy1 @admin what's the update here?

                      1 Reply Last reply Reply Quote 0
                      • V
                        Vypy1 last edited by

                        @admin what's the update on this?

                        1 Reply Last reply Reply Quote 0
                        • A
                          admin @Vypy1 last edited by

                          Hi @Vypy1

                          Historical data API is only able to fetch 500 records at a time.

                          Meaning if you are trying to fetch the data using below parameters it will fetch data from 13th Oct 2020 to 18th Oct 2022. Because the Gap between fromdate and todate is > 500 days, it is designed to return latest 500 values as follows = todate-500 days.

                          {
                          "exchange":"NSE",
                          "symboltoken":"11821",
                          "interval":"ONE_DAY",
                          "fromdate":"2020-08-09 11:00",
                          "todate":"2022-10-18 11:40"
                          }

                          Hope this helps to clarify.

                          V 1 Reply Last reply Reply Quote 1
                          • V
                            Vypy1 @admin last edited by

                            @admin Thanks admin, I tried this and it works, however earlier I could fetch 2000 records for daily candles. Even this page says I can fetch 2000 days for 1 Day candles

                            https://smartapi.angelbroking.com/docs/Historical

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post