[Windkit] 10min data to tab

[Andrea Vignaroli Ørsted]

Hello ex-colleagues!

I am just being lazy and wanted to take advantage of Windkit's histogram plots, windrose plots and weibull fitting.

 i can't remember/find if the conversion from 10min statistics (well organized in a Pandas dataframe) to a tabfile is currently part of the package or not yet.

My regards to everyone in RAM!

Andrea

[Neil Davis]

Hi Andrea,

We don't have a pandas to bwc, but we have an XArray Time-series dataset to bwc converter. You can find the docs for that at: https://docs-wasp.ramtt.xyz/windkit/io/wc_autogen/windkit.binned_wind_climate.bwc_from_timeseries.html#windkit.binned_wind_climate.bwc_from_timeseries

 

It would be interesting to know the structure of the pandas data that you have, as we are looking to add more functionality to the time-series handling this year, and creating an XArray tswc from pandas is probably something that would be good for us to support.

[Andrea Vignaroli Ørsted]

Hi Neil, thanks for the heads up. I missed the xarray to bwc conversion functionality. I will give it a try.

The format of the pandas dataframe is very simple and it's something i came up with. The format of time series in Ørsted is standardized only at the database level.

                       WS_35m      WD_35m  ...     WD_285m   SD_285m
timestamp                                  ...                      
2022-08-10 05:00:00       NaN         NaN  ...         NaN       NaN
2022-08-10 05:10:00  1.579420  111.637028  ...  200.177297  0.271540
2022-08-10 05:20:00  1.589379  132.519061  ...         NaN       NaN
2022-08-10 05:30:00  1.740071  133.400359  ...  161.574321  0.226531
2022-08-10 05:40:00  1.764799  143.742655  ...  163.586885  0.169169
                      ...         ...  ...         ...       ...
2023-01-31 23:10:00       NaN         NaN  ...         NaN       NaN
2023-01-31 23:20:00       NaN         NaN  ...         NaN       NaN
2023-01-31 23:30:00       NaN         NaN  ...         NaN       NaN
2023-01-31 23:40:00       NaN         NaN  ...         NaN       NaN
2023-01-31 23:50:00       NaN         NaN  ...         NaN       NaN

[25170 rows x 9 columns]

[Neil Davis]

Great, thanks. Hope the XArray to BWC works for you.

[Omer Khan]

Hi @Andrea Vignaroli Ørsted & @Neil Davis,

@Andrea Vignaroli Ørsted did you manage to make binned_wind_climate from pandas time-series datafrme?

I tried making xarray dataset from pandas dataframe and then using the xarray dataset to "windkit.binned_wind_climate.bwc_from_timeseries", however I get some errors.

My pandas dataframe (DF) has two columns wind_speed & wind_direction and index as time (see table below).

time				wind_speed		wind_direction
2012-10-30 23:00:00		5.11			209.1
2012-10-30 23:10:00		5.42			244.6
2012-10-30 23:20:00		5.22			220.6

Minimal reproducible example of my code with dummy data is:

import windkit
import pandas as pd
import random
import xarray as xr

# Function to make dummy dataframe
def dummy_df(size):
    df = pd.DataFrame(zip([random.uniform(0, 25) for i in range(size)], [random.uniform(0, 360) for i in range(size)]),
                      index=pd.date_range("01-01-2023 00:00:00", periods=size, freq="10T"), columns=['wind_speed', 'wind_direction'])
    df.index.name = 'time'
    df = np.round(df, 2)
    return df

# Pandas dataframe
DF = dummy_df(6*24*365)

# dataframe to xarray dataset
DS = xr.Dataset.from_dataframe(DF)

# Windkit binned wind climate from timeseries
bwc = windkit.binned_wind_climate.bwc_from_timeseries(DS, ws_bin_width=1, nwsbin=35,
                                                      nsec=12, normalize=True, crs=None)

The errors that I get are,

WindClimateValidationError: validate found 4 errors 
1. Missing coordinate: south_north
2. Missing coordinate: west_east
3. Missing coordinate: height
4. Missing coordinate: crs

I tried manually adding these missing coordinates (dummy values) to xarray dataset (DS) with following code.

# Manually making missing xarray coordinates for DS

height = xr.DataArray([90], dims='point', coords={'point': [0]})
DS.coords['height'] = height

south_north = xr.DataArray([0], dims='point', coords={'point': [0]})
DS.coords['south_north'] = south_north

west_east = xr.DataArray([0], dims='point', coords={'point': [0]})
DS.coords['west_east'] = west_east

crs = xr.DataArray([0], dims='point', coords={'point': [0]})
DS.coords['crs'] = crs

After this, I get new error that is,

KeyError: 'crs_wkt'

I also manually added a DataArray "crs_wkt" to the DS dataset but it doesn't work.

Any pointers where things are being done wrong?

Thank you 🙂

[Rogier]

Hi Omer, your code is nearly working. The only part that is missing is assigning the coordinates and the crs (coordinate reference system). If your site has latitude=56 deg N and longitude=10 deg E and is at 100 m above the surface, you could do something like this:

DS = DS.assign_coords({"west_east":10,"south_north":56,"height":100.})

windkit.add_crs(DS, 4326)

# Windkit binned wind climate from timeseries

bwc = windkit.binned_wind_climate.bwc_from_timeseries(DS, ws_bin_width=1, nwsbin=35,

                                                      nsec=12, normalize=True, crs=None)

The `add_crs` function adds the coordinate reference system, in this case a geographic coordinate system which is denoted with the EPSG code 4326. If you use another coordinate system you can find the corresponding code usually here:
https://epsg.io/

or you can create you own `crs` object using the pyproj library, which windkit relies on for projections.

Edited March 30, 2023 by Rogier

[Omer Khan]

Hi @Rogier,

Wow! It worked and BWC dataset was generated.

However now I get another error (pasted below) when I use this BWC dataset to get weibull fit using "windkit.weibull.weibull_fit(bwc, include_met_fields=None, keep_attrs=True)".  

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File <timed exec>:1
File ~\AppData\Roaming\Python\Python39\site-packages\windkit\weibull.py:61, in weibull_fit(bwc, include_met_fields, keep_attrs)
     57 # TODO: Add include met_fields here
     59 wb = update_var_attrs(wb, _WEIB_ATTRS)
---> 61 return update_history(wb)

File ~\AppData\Roaming\Python\Python39\site-packages\windkit\metadata.py:460, in update_history(ds)
    458 current_utc = datetime.utcnow().replace(microsecond=0).isoformat()
    459 package = __name__.split(".")[0]
--> 460 function_call = inspect.stack()[2].code_context[0][:-1]
    462 if "=" in function_call:
    463     function_call = function_call[function_call.index("=") + 1 :]
    
TypeError: 'NoneType' object is not subscriptable

MRE as follows.

import windkit
import pandas as pd
import random
import xarray as xr

# Function to make dummy dataframe
def dummy_df(size):
    df = pd.DataFrame(zip([random.uniform(0, 25) for i in range(size)], [random.uniform(0, 360) for i in range(size)]),
                      index=pd.date_range("01-01-2023 00:00:00", periods=size, freq="10T"), columns=['wind_speed', 'wind_direction'])
    df.index.name = 'time'
    df = np.round(df, 2)
    return df

# Dummy pandas dataframe
DF = dummy_df(144)

# Dataframe to xarray dataset
DS = xr.Dataset.from_dataframe(DF)

# Assigning coords to dataset
DS = DS.assign_coords({"west_east":0,"south_north":0,"height":90.0})

# Adding CRS to dataset
windkit.add_crs(DS, 4326)

# Windkit binned wind climate from timeseries
bwc = windkit.binned_wind_climate.bwc_from_timeseries(DS, ws_bin_width=1, nwsbin=35,
                                                      nsec=12, normalize=True, crs=None)

# Fitting windkit weibull
weibull = windkit.weibull.weibull_fit(bwc, include_met_fields=None, keep_attrs=False)

What possibly could be wrong here? 😅

BR

Omer

[Neil Davis]

I can take this one. There is a challenge here when you are using the Python REPL. If you use ipython, a jupyter notebook, or call it as a script the code will work. I have an issue pending about giving a better error message here.

[Omer Khan]

Hi @Neil Davis,

Appreciate your prompt reply. I was using jupyter lab, it didn't work there but yes it worked in Spyder (Ipython).

Thanks to you and your team for publishing this amazing library. Highly appreciate support on this forum and the efforts for this library. 🙂

BR

Omer

[Neil Davis]

Hmm good to know about Jupyter lab, will take a look at that as well to see if it is the same issue as the REPL. We haven't advanced past the notebook interface yet 🙂

Glad to hear that the package is so helpful. We have already started discussing an interface for helping convert pandas based timeseries to our xarray time-series format, so hopefully this workflow will be easier in the future.