Topic: Webservices

There is quite a lot of talk of using webservices in order to get LRCs using NBN Gateway data.  However, it is unknown how many LRCs are actually interested in webservices implementations.  When I talk to LRCs face to face, there are quite a few differing opinions.  Of course some LRCs already use webservices in order to include Gateway data in the enquiry output.  Others are reluctant to do so for a variety of reasons.

It would be really helpful if LRCs were able to post on here whether or not it is something they are interested in.  Perhaps you could just leave a note saying something like "yes - because..." or "no - because..."

Many thanks,

Tom Hunt - ALERC National Coordinator

Re: Webservices

No - because from past experience it was unusable. This may change when v5 of the Gateway is rolled out fully.

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

3

Re: Webservices

Yes, interested but probably not skilled enough at present.  Due to our patchy county recorder network and restricted staff time here several key national datasets would be useful to interact with via webservices.

Cambridgeshire


Tom wrote:

There is quite a lot of talk of using webservices in order to get LRCs using NBN Gateway data.  However, it is unknown how many LRCs are actually interested in webservices implementations.  When I talk to LRCs face to face, there are quite a few differing opinions.  Of course some LRCs already use webservices in order to include Gateway data in the enquiry output.  Others are reluctant to do so for a variety of reasons.

It would be really helpful if LRCs were able to post on here whether or not it is something they are interested in.  Perhaps you could just leave a note saying something like "yes - because..." or "no - because..."

Many thanks,

Re: Webservices

Yes Cofnod are interested if it is possible to make the process of requesting permissions more straightforward.

Aisling May
Data Manager - Cofnod   www.cofnod.org.uk

Re: Webservices

Yes - definitely. But we (in Wales) have been told that we should be getting permission from all data suppliers before any commercial use. Have others already achieved this? If so, how? I think this is something we really need your help with Tom.

Adam

Adam Rowe
Manager, South East Wales Biodiversity Records Centre (SEWBReC), 13 St. Andrew's Crescent, Cardiff, CF10 3DB. T: 029 2064 1110

Re: Webservices

Yes eventually, as supplementary info for enquiries, but development time a barrier at the moment, as well as permissions. It's not a priority.
Major national datasets we get permission for, where we are happy with the data, we are likely to download copies of from the Gateway and not access via web services.

Rather than other orgs' data I am more interested in accessing our own data via web services to make it accessible to people from our own website - particularly if we can do that at a finer resolution than our public setting on the Gateway to get around the download issue on v5 (I think this is technically possible but not sure.) i.e. so our data is viewable at 10km on data.nbn.org.uk but at tetrad level on cbdc.org.uk. And even more so if we can use it to get species lists, in taxon groups, by parish - as we presently have a separate system to do this on lakelandwildlife.co.uk, which we can no longer update.

Teresa Frost
WeBS National Organiser, BTO. NFBR Council Member.
ex-ALERC/CBDC/KMBRC

Re: Webservices

Hi Adam,

There are a couple of ways of approaching this problem.  The first is to do some gap analysis, which the NBNT have been able to fund in past.  I am not sure whether there is currently an oportunity for LRCs to do this.  The other thing to think about is that it might rely on Defra money that restricted to England only.  Paula will be able to help with this.  Once complete, you will then have a list of people you can send a generic data access request to.  We did this at RECORD, but we need volunteer help in order to get the request sent to everyone.

The other approach is to wait until the new data access controls are launched when you will be able to use geographical filters in order to see what access requests you need to submit for your area.  I am not sure if you will be able to submit several requests with a single click, which would obviously reduce the time burden.  Again, I am sure Paula can help with this.

Thanks for all the responses so far on this topic.

Tom Hunt - ALERC National Coordinator

Re: Webservices

Tom wrote:

submit several requests with a single click

Pretty sure this will be in/was planned for v5 of the gateway

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

Re: Webservices

Re permission requests - am a bit lost with Gateway - but can organisations be a member of organisations (I think perhaps... in v5???)
So Alerc could contact e.g. the caddis fly scheme (who want all LRCs to use the data) and then they can give access to Alerc in one click - and thus all Alerc member LRCs.
Alternatively if the data is public at full res, Alerc (i.e. you Tom sorry) could maintain a list of datasets where permission has been granted to LRCs, so we know we can use.

Teresa Frost
WeBS National Organiser, BTO. NFBR Council Member.
ex-ALERC/CBDC/KMBRC

10

Re: Webservices

Hi Teresa,

I will endeavour to find an answer to your question.  I would suggest that if this is possible the only problem might be that data providers may wish to provide access to some LRCs and not others.  I suppose there is also the question of whether or not LRCs are happy having something like this administered by a third party.  Anyway, we can cross this bridge when we come to it.  Let's find out if it's possible first.

Regards

Tom Hunt - ALERC National Coordinator

Re: Webservices

Hi
The ability to apply for access to multiple datasets with a single request will be available in Gateway 5.  There is currently no mechanism for applying for permission via the Gateway but we agreed at a Trustees meeting yesterday that this should be developed, I will keep you posted about details and timescales.
It is not possible (even in Gateway 5) for organisations to be members of a superorganisation and for the superorganisation to apply for access on behalf of all member organisations.  It has been discussed before and if it is something that would really help lead to increased use of data from the Gateway/web services then we could scope this out, however this kind of arrangement wouldn't deliver the benefit of LRCs being able to apply for access just to the data within their boundary, and I know that at least some national schemes would prefer to grant full access just to a subset of their dataset.
The funding for the gap analysis work was via Defra so I'm sorry but it was just for English LRCs.  However, Graham wrote a query to produce the data for the gap analysis for multiple LRCs at the same time, so it would probably not take long to run this again for anyone else (not just in England) who's interested.  The data summarises all records within the LRC's boundary broken down to show the number of records at the best available level of access and at the LRC's current level of access, and also shows this for records of designated taxa.  The funding we have available now would be for some kind of development (preferably by LRC staff/ex LRC staff as they know the requirements best) to help other LRCs adopt web services, e.g. a GIS tool or web-based solution.
Best wishes
Paula

Re: Webservices

Yes, definitely interested. It would be good to see a few real open source implementations in, say, PHP to act as a guide and tutorial. For example, it would be useful to see a site that allowed the user to enter an area of interest (an LRC boundary for instance) and a date and it would give details of all new and updated datasets for that area, display it on a map, and have a facility to request permission and download those datasets. In other words, something that demonstrates most of the WS capabilities (including v5). The full source for this example implementation would be released as open source.

I've always found the current WS API docs utterly inscrutable. E.g. http://data.nbn.org.uk/Documentation/We … /Grid_Map/

Part of the problem there I think is that SOAP is a pretty heavyweight, 'enterprisey' approach to web services, whereas the JSON/REST approach is much simpler and more accessible are therefore favoured by web developers. E.g.: http://blog.feedly.com/2009/03/03/jsonrest-vs-xmlsoap/

Re: Webservices

I think Charle's idea is excellent. I can see it going to be a lot of work keeping track of which Gateway datasets we have access to for BIS reporting and webservices and then applying for access for new datasets.

Janet Imlach (Manager)
Powys and BBNP Biodiversity Information Service

14

Re: Webservices

Please see this news release from the NBNT explaining forthcoming technical developments:

New NBN Web Services

NBN web services provide a way of embedding data and maps from the NBN Gateway into other systems and websites.  Web services are used by a wide range of NBN partners for many purposes; to inform environmental decision making, to support the volunteer recording community or simply to help people find out about the wildlife that has been recorded

Tom Hunt - ALERC National Coordinator

Re: Webservices

We would be interested in using the new REST service. I did a search on NBN website and come across the REST api for existing data (http://gis.nbn.org.uk/arcgis/rest/services/siteBoundary/siteBoundaries/MapServer/?f=help and http://gis.nbn.org.uk/arcgis/rest/servi … p;f=pjson).

It seems that ARC provide a tool called ArcObject which will help parsing some of the REST/JSON data. But you need to have a copy of Arc GIS to use ArcObject.

rECOrd

16

Re: Webservices

Great.  Have you received a communication from Paula recently?

Regards,

Tom Hunt - ALERC National Coordinator

Re: Webservices

Hi Tom,

Yes we have and we should be responding next week. Is that mean it's possible to access the REST service early?

Cheers,
Luck

rECOrd

Re: Webservices

Hi Luck
Yes, the developer/s who take part in the trial project between June-September will have early access to the RESTful web services, before they are documented and released more widely.  Once the RESTful services are released there will be further opportunities for support to help other LRCs implement them.
Thanks
Paula

19 (edited by BDeed 09-05-2013 12:30:01)

Re: Webservices

In reference to the original question.

Yes - we already make extensive use in combining the data with our own for services and are very interested in further implementing v5 services as a matter of priority.

Picking up on Charlies point above i agree that currently the services are off putting due to the amount of down time (e.g. yesterday and today!) hopefully this will be something that will be fixed in v5.

Ben Deed - LERC Officer
Merseyside BioBank

20

Re: Webservices

The NBNT have produced some new information for those of you who are interested in developing a webservices implementation.  You can click here https://data.nbn.org.uk/Documentation/W … _Examples/ to see tutorial examples.

Regards,

Tom Hunt - ALERC National Coordinator

Re: Webservices

How very co-incidental..

Ben Deed - LERC Officer
Merseyside BioBank

Re: Webservices

Still nothing for desktop applications though?

Ben Deed - LERC Officer
Merseyside BioBank

Re: Webservices

Thanks Tom,

Looks simple enough thanks. Is there a way to provide an API key as part of the request so that the results reflect additional permissions we have been granted? Apologies if this has been covered on a different post. I know this is possible with the SOAP implementation from memory. The documentation doesn't seem to include a reference to 'API key'.

24 (edited by matt.debont 26-06-2014 10:45:19)

Re: Webservices

Hi, I am one of the developers for the gateway and would like to try and answer some of the questions you are having,

BDeed wrote:

Still nothing for desktop applications though?

The NBNClient portions of the examples are exactly the same for both creating a web service of your own or creating a desktop client in this case, the only difference here being that you would be outputting that results from the client to a desktop client rather than representing them on a web page for example.

The Java and C# examples should be enough to get you started with a client of your own in any language (https://data.nbn.org.uk/Documentation/W … BN_Client/) I am happy to answer questions here however.

bazmole wrote:

Is there a way to provide an API key as part of the request so that the results reflect additional permissions we have been granted? Apologies if this has been covered on a different post. I know this is possible with the SOAP implementation from memory. The documentation doesn't seem to include a reference to 'API key'.

V5 of the gateway does not use API keys to grant access any more, you need to log in to the web services and from that point onwards you will get that users permissions applied to the responses form the REST services. There are examples of how to do this in the NBN Client examples as above, but the authentication is handled by cookies, so when you log in you will get a cookie in return, you need to include this cookie with all future calls and you will then be given the appropriate responses.

The gateway itself runs directly off the web services, so in theory anything you can do on the web site you can do through the REST services, the documentation is lacking in some places and I freely admit that, but we are working to improve them and in the mean time feedback is always useful.

I hope that helps and I am happy to continue to answer questions or at least help push you in the right direction.

Matt Debont
Application Developer
Joint Nature Conservation Committee, Monkstone House, City Road, Peterborough PE1 1JY, UK

Re: Webservices

Thanks Matt,

That makes sense :-)

Re: Webservices

Cheers Matt,

Shortly after this post i did actually manage to make some progress, i'm working with VB .NET so not too much of a stretch. Unfortunately, pretty limited by both time and the fact i'm not a developer! Still progress is progress and when i have something functional i would be happy to share over on the similar NBN thread. I today have a limited working application that returns TaxonObs for a grid square using the LRC login, Huzzah!

p.s. i/the json deserializer is falling slightly on special characters in the Authority field, any tips appreciated!

Ben Deed - LERC Officer
Merseyside BioBank

Re: Webservices

BDeed wrote:

p.s. i/the json deserializer is falling slightly on special characters in the Authority field, any tips appreciated!

Sounds like it would be a Unicode issue, there are a lot of authorities that contain letters with accents, etc... I don't know much about the VB.Net JSON deserializer so I can't offer much more help there I am afraid, but the issue is quite likely to be a unicode issue

Matt Debont
Application Developer
Joint Nature Conservation Committee, Monkstone House, City Road, Peterborough PE1 1JY, UK