About lortie

ecologist & runner.

Collaborative and open science writing

As we collectively move to platforms that support better reproducibility and open science, a few tiny challenges persist. Reference management. LaTex with BibTex is great, but at times, team members are interested less in reproducibility and more in just sharing the libraries. We recently faced this challenge because we were collaboratively writing a very long white paper and each of us worked in a different management ecosystem in spite of using GitHub to control the versioning and collaboration in the writing.

Here are some resources to support a decision. Anecdotal research similar.

List with satisfaction scores

Refworks, Easbib, Endnote, and Mendeley look promising.

Good contrast here including discussion of Zotero.

Gradhacker review here of offerings.

Writing in google docs collaboratively use paperpile.

Writing in RStudio use Zotero.

Summary

Great lists of pros and cons out there. Based on the various lists, I vote for key criteria as a. cloud storage, b. can use in RStudio easily, c. allows me to share library with collaborators for a given paper.

The three competitors seem to be Refworks, Mendeley, and Zotero.
Now, need to give them a head-to-head test shortly.

 

Seven steps for turning camera trap photos into useful datasets: manual processing workflow

Seven magical steps into a dataframe.


By Nargol Ghazian

This is a summary of the protocol I have been using for that past few months to process all the amazing camera trap photos from the Mojave National Preserve and the Carrizo National Monument. After reading a few papers on cam trap processing and exploring the CamtrapR package, the best approach would be to create your datasets manually as no other program is able to automatically detect animals for you. This method also ensures that you obtain the best dataset for the statistical analysis you wish to perform. This seven step guide should give you a quick rundown on how to get started with processing and maintaining a good workflow.

 

  1. Start by naming your columns as below. These heading are best suited for this project but yours can include more columns.
1.Year We are working on the 2017 images
2.Region MNP is Mojave and CNM is Carrizo
3.Site Mojave or Carrizo
4.Calendar date The date the picture was taken in dd-mm-yr. I like to do the pictures belonging to the same date for each photo rep in order. If the date is wrong, don’t worry too much, just do it all as one for the last date of the particular week you are working on
5.Microsite Carrizo is shrub or open, 3 weeks for each. Mojave is Buckhorn or Larea, also 3 weeks for each.
6.Day This goes in a 1,2,3..n order
7.Rep This refers to the camera trap station. There are 10 stations per microsite. For example you might have four pictures for the same day in station #2 of open, so you would write 2 four times: 2,2,2,2 each corresponding to an image
8.Photo Rep A continuous number starting at 1 and continuing until you’ve finished processing all your pictures for the particular site
9.Animal The animal in a hit photo. The most common are rat, rabbit, squirrel, fox, lizard and sometimes bird. There are times where you might have to guess. If it’s really hard then write ‘unidentifiable’. If it’s a false hit leave it blank.
10.Animal. Capture binary 0 = false hit, 1 = animal present
11.Time block Look at the timestamp. Is the photo taken at night, noon, afternoon or in the morning. If the timestamp is wrong, guess based on the darkness or lightness.
12.Night. Day Based on whether it’s dark or light.
13.Actual time Actual time written on the photo. Let’s hope it’s the correct timestamp J
14.Observations If you see absolutely anything interesting in the photo, note it! Otherwise leave this column blank. I usually write ‘x2’ or ‘x3’ if there is more than one animal in the photo. Sometimes I write ‘eyes visible’ if it’s dark and you can only tell the presence of the animal from its shining eyes (rats usually)
15.Temp of positive This is noted on the picture in Fahrenheit or Celsius. Whatever unit is shown, note it in your meta-data. If you’ve been working with one unit, and a certain photo rep has a different one, just use a converter to convert to the units you’ve been using for that particular photo rep.
16.Week This is either 1, 2 or 3 since there are only 3 weeks per microsite. This column is super important because sometimes the datestamps are wrong but at least the week of sampling is correct

*Note: The only time we actually fill in anything for columns 9 and 11-15 is when we have a “hit” and there is an actual animal.

  1. Since each row corresponds to a particular image, I liked to start with at least 100,000 just so I don’t have to go back and paste more rows. You can delete the extra rows when you’re done processing. For year, region, site and microsite write the correct label in the cell right below your column heading. Select the cell, then using control+C input the number of cells you want by using a colon in the area below the clipboard in Excel. For examples if you are in row B, it would be written as B2:B100000 then click enter.
  2. You can use the above method for the date as well. I input the date as I go along. For example if the photos start on 2017-05-22 I paste 100 thousand of that date. Obviously this is far too many, so once all the images for that particular date is finished, I control+C from the last cell and enter the new day for up to 100 thousand and so on until you are finished for the week and can delete the extra cells.
  3. Days work exactly like calendar dates.
  4. When it comes to rep, select all the image belonging to a particular date in your particular station of cam trap files and click properties. This tells you how many images you have for that date, in that station. Use the above method discussed to paste as many cells needed for that station. Keep a calculator handy because you would need to determine your ending cell by adding the number of images to the current cell you’re on.
  5. Photo rep is just a continuous number, as already mentioned in the chart. It’s basically the total of all the images processed for either Carrizo or Mojave. Use the ‘fill’ button in excel for numbers in a series to do this step.
  6. If you come across a cool photo, make sure to copy and paste it in a different folder. This includes two animals together, animals fighting, clear photos of animals etc. (use your judgement). Do not cut and paste!

 

 

How to download data from @HOBODataLoggers using a Mac #openscience #opendata @apple

Connecting most peripherals to a Mac is typically a snap. However, about two years ago, updates to OSX introduced to challenges to connecting Onset Hobo Micro Data Loggers to initialize and then download stored data. I decided to finally work through these challenges instead of switching machines. This may seem trivial, but it was a bit finicky; so, here are the steps quickly summarized.

Configurations: any version of OSX 10.8 and higher likely needs these steps particularly if your Onset product uses a serial port for communication.  The steps listed below were developed on a late 2012 iMac running 10.13.1 OSX (High Sierra).

Steps to connect

  1. Install Java Version 7 or higher (I installed Version 8 Build 151) because this is a dependency in later step.
  2. Reboot.
  3. You will need a usb to serial adapter for older Onset Stations. The one provided with loggers is manufactured by Tripp Lite and entitled Keyspan High Speed USB to Serial Adapter (USA-19HS). This step was a nightmare. You must install the correct driver and reboot. I tried earlier versions of the adapter and failed. Here is the manufacturer support page with driver downloads.  Continue forward with steps, but if you have everything plugged in ready to go and cannot see logger in HoboWare app, return to web and search for driver beta version and install. I had to do this, and I used driver version 4.0b4 (beta version) – not the version 3 from site. Here is a good explanation (need to go to Tripp Lite overview page and search for first three characters of unit, i.e. ‘USA’). If you do any updates to OSX, you also have to reinstall the driver.
  4. Install HoboWare Free Version (I prefer over the Pro Version that is provided with loggers because it is more updated more frequently and seems to run more smoothly). I used Version 3.7.13 here.
  5. Connect adapter to station then to serial-to-usb adapter and launch HoboWare.  You are now ready to either initialize or download if you have already deployed loggers.
  6. Initialize via device > launch. This is pretty self-explanatory.
  7. Download was a bit less obvious to me. Use device > readout. Allow readout to complete (progress bar pops up).
  8. Once complete, data are read into this instance of HoboWare as a project not saved. I recommend doing plot right away (see below).
  9. You can now use file > save datafile to locally save a file ending in .dtf. This is not ideal for me. The save project saves data, thumbnail, details, and plots for future reading by app. Again, do not prefer because I work in R. See final step if you want to work outside ecosystem.
  10. If you prefer to save as .csv, you first select the plot option after readout because this writes workspace to .csv. Then go to file > export data table and you have a nice clean dataframe for additional wrangling. Before you move on to next logger, ensure you close project because it can retain former instance of plots and datable even after you do a readout from new unit.

Now, you are ready to explore some microclimate for your sites!

Data logger discoveries

The first micronet study is now complete. Cuyama Valley was instrumented using Onset micro-stations with temperature and soil moisture sensors for two full seasons. I made some very important discoveries in wrapping it up this autumn.

 

  1. Cables are heavy.
  2. Animals are much more active in the summer.
  3. Cables are yummy to animals.
  4. Deploying loggers with sensors is a magnet for new burrows.
  5. Heat cooks batteries.

Here is hoping the data recovered are just as fascinating. Honestly, I am tempted to do a wire-addition experiment. Observation suggests that there is a very real magnet effect of wires.

 

Posted in fun

Cam trapping: ten simple steps to process all those amazing photos

Workflow

  1. Read some cam trap papers.
  2. Check camtrapR package and see what it does to decide if it suits your specific needs.
  3. Open folder for each site, each day, each rep.  Do a folder ‘get info’ to count # of total pics.  These are your ‘reps’ within reps, i.e. literally total number of snapshots (or use command line to get dir info for all your photo data).
  4. I would honestly just paste 0’s all the way down because many will be ‘false hits’.
  5. Then, open them all up and scroll through.
  6. Every time a positive hit, overwrite 0 in ‘animal.capture’ vector and in ‘animal’ vector record what it is.
  7. Also, copy all positive hits photos into a separate folder for additional analyses.  Use a folder structure or ID system that keeps track of the place and time that photo was from.  For instance, have a folder entitle positive-hits for each site, day, location, rep or aggregate into a single positive hit folder but use a mechanism to ensure we know where/when photo was taken. Do not cut and paste, copy. This is a backup mechanism for additional analyses and sharing data.
  8. We also want to know when animals are most active, or not; and hence, check timestamps and paste down in that column too. Ideal is actual time but morning, afternoon, night is absolutely adequate and more rapid if we cannot automate the scraping using R-package.
  9. If timestamp is incorrect, do a light-dark assessment to code as night or day – this is a very rapid process.
  10. Record observations if more than one animal or if the same animal was recaptured from previous instance. Record anything of note ecologically to calibrate the quantitatives and link photo-capture processing to data mapping/translation. The goal is to accurately map photos onto numbers that represent the dynamics of the system in study.

Outcomes

The goal is have to have both an evidence folder of positive hits and a dataframe that can then be wrangled to estimate relative efficacy of sampling, frequencies of different animals, spatiotemporal dynamics, and differences between structured treatments in the implementation of trapping.

Meta-data for manual processing spreadsheet workflow

Attribute is the column headers.

attribute description
year we have many years for Carrizo (evil laugh) so good to list here in a vector
region MNP for Mojave, CNM for Carrizo
site if you have more than one site, put name of site
calendar date dd-m-year
microsite larrea, buchhorn, ephedra or open depending on region
day this is census day, 1,2,3, to however many days sampled
rep if more than one rep per day
photo rep just cut and paste to total number of photos each cam took on one day, could be 10 to 10000
animal.capture binary 0 = false hit, 1 = animal present
animal list animal as ‘none’ if false hit, then animal name if one was there
timeblock with animal telemetry work, morning, afternoon, night is usually sufficient 6am to noon, noon to 6pm, them night time
night.day back up if timestamps are incorrect – just do by night and day using light and darkness in photos. very quick
filename.ID.positive.hits
optional depending on your filing system, copy all positive hits to a separate folder. somehow, keep track of positive locations and times for subsequent analyses
observations
record observations of anything ecological that pops such as if there was another animal in the photo OR if it was the same animal repeatedly recaptured

MSc or PhD for Canadians to do research in desert ecology or open science in California

Great news, we have had some funding come through for some research in California.

Two options, MSc or PhD.

Desert ecology research

The primary focus of the research is exploring how we might better use positive interactions between plants for restoration and management of arid systems. In particular, we want to examine influences on other taxa such as insects (including pollinators), endangered animal species (such as leopard lizards and kangaroo rats – cute), and on community biodiversity dynamics.

Details

Graduate-level research with The Nature Conservancy in Carrizo National Monument of positive-plant animal interactions.

GPA for YorkU Biology is A-, A.

Need to be able to drive.

Competent in R.

Admission Requirements

Open science research

Graduate-level research on open scientific synthesis. The goal is to explore existing data in high-stress ecosystems such as deserts and do synthesis. Data aggregation, systematic reviews, and meta-analyses to explore the importance of foundation species and biodiversity. This is a unique opportunity because this person can also collaborate with NCEAS to explore teaching open scientific synthesis, develop materials, and do research with the process of doing open science for synthesis.

Details

Excited about open and team science.

Competent in R.

Excited to work with big data, access data repositories, and do synthesis.

Excited to become an educator and contribute to positive change by developing materials (code, packages, guidelines, etc) that use these approaches.

Same admission requirements as first position.

Start date is Sept 1, 2017 for both/either opportunity.

http://futurestudents.yorku.ca/graduate/apply-now

I recommend you pop me an email too if you are interested: lortieATyorku.ca

How we use a handheld soil moisture probe to supplement in situ plant ecology sampling

It is best to deploy loggers with appropriate sensors to capture an environmental signal within a set of study sites. Nonetheless, when actively sampling for plant-plant interactions dynamics,  an estimate of soil moisture at that particular point in time and space precisely is useful (at least as a covariate). We use the Delta-T SM-150 handheld unit to complement our long-term logging arrays.

Here is a brief summary of the settings/methodology we use.

Method

  1. Push right button to activate unit.
  2. Repeatedly depress right button to cycle through modes until you reach ‘organic’.
  3. Insert probe into ground and ensure that metal conductors are fully embedded in ground with ceramic/plastic unit flush with ground surface.
  4. Left button to measure. Typically, it should take only 1-2 seconds.
  5. Avoid rocks and voids in the ground when inserting probe.

Comments: Ranges you can expect at least in arid and semi-arid systems we have tested within California are between 1-40% but most frequently < 10%.  The unit is durable, and the control unit is ‘water resistant’. However, when the controller gets wet in the rain, it stops working until it drys out again (typically at least a day later). The cable is not that robust, and to be safe, we insert/push the sensors into the ground using the ceramic casing.