performance - Fill Matrix fast - R -


i started prediction models in r , have performance question:

currently have 2 datasets sunset , sunrise data.

i computed them years 2011 , 2012 (daily data). real data contains first half of each month. tried integrate sundata in data frame loop:

library(lubridate) library(maptools)

sunrise functioin from: https://gist.github.com/hilaryparker/2a83ca521353e8478c92

sunrise.set <- function (lat, long, date, timezone = "utc", num.days = 1)   {   lat.long <- matrix(c(long, lat), nrow = 1)   day <- as.posixct(date, tz = timezone)   sequence <- seq(from = day, length.out = num.days, = "days")   sunrise <- sunriset(lat.long, sequence, direction = "sunrise",                        posixct =true)   sunset <- sunriset(lat.long, sequence, direction = "sunset",                        posixct = true)   ss <- data.frame(sunrise, sunset)   ss <- ss[, -c(1, 3)]   colnames(ss) <- c("sunrise", "sunset")   return(ss) }   datetime<-c(seq(from = as.posixct("2011-01-01 00:00"), = as.posixct("2011-01-19 24:00"), = "hour"),seq(from =as.posixct("2011-02-01 00:00"), = as.posixct("2011-02-19 24:00"), = "hour")) train<-data.frame(datetime,1)  date<-force_tz(as.posixlt(seq(as.date("2011/01/01"),as.date("2011/02/28"), = "day"), format="%y/%m/%d",tzone="america/detroit"),tzone="america/detroit") sunrise<-data.frame(date,sunrise.set(38.889931,-77.009003,"2011/01/01", timezone = "america/detroit", num.days = 59))   sunrise.train<-as.posixlt(na) sunset.train<-as.posixlt(na) (i in 1:length(train$datetime)){  for( j in 1:length(sunrise$date)){print(i);print(j)     if(as.date(as.posixlt(train$datetime[i]))==sunrise$date[j])  {sunrise.train[i]<-sunrise$sunrise[j];sunset.train[i]<-sunrise$sunset[j]} }}  train.modified<-data.frame(train,sunrise.train,sunset.train)

unfortunately loop small, takes 2-3 hours compute loop full data. there faster way "fill" training data frame suitible data?

thank much!

edit:changed code appearance

you want simple merge

update:

# reduce data testing train <- train[1:100,] sunrise <- sunrise[1:10,]  yf <- function() { (i in 1:length(train$datetime)) {   (j in 1:length(sunrise$date)) {     if (as.date(as.posixlt(train$datetime[i])) == sunrise$date[j])  {       sunrise.train[i] <- sunrise$sunrise[j]       sunset.train[i] <- sunrise$sunset[j]     }   } } list(sunrise.train, sunset.train)   }  system.time(r <- yf()) # ~ 3 sek 100 x 10  train.modified <- data.frame(train, sunrise = r[[1]], sunset = r[[2]]) # results

using data.table:

require(data.table) setdt(train) setdt(sunrise)  # reformat dates equal formats train[, date := as.date(as.posixlt(datetime))] sunrise[, date := as.date(date)]  myrez <- merge(train, sunrise, = "date", all.x = t) myrez <- myrez[, -1]  setdt(train.modified)  head(train.modified) #              datetime x1             sunrise              sunset #1: 2011-01-01 00:00:00  1 2011-01-01 07:26:39 2011-01-01 16:56:37 #2: 2011-01-01 01:00:00  1 2011-01-01 07:26:39 2011-01-01 16:56:37 #3: 2011-01-01 02:00:00  1 2011-01-01 07:26:39 2011-01-01 16:56:37 #4: 2011-01-01 03:00:00  1 2011-01-01 07:26:39 2011-01-01 16:56:37 #5: 2011-01-01 04:00:00  1 2011-01-01 07:26:39 2011-01-01 16:56:37 #6: 2011-01-01 05:00:00  1 2011-01-01 07:26:39 2011-01-01 16:56:37 head(myrez) #              datetime x1             sunrise              sunset #1: 2011-01-01 00:00:00  1 2011-01-01 07:26:39 2011-01-01 16:56:37 #2: 2011-01-01 01:00:00  1 2011-01-01 07:26:39 2011-01-01 16:56:37 #3: 2011-01-01 02:00:00  1 2011-01-01 07:26:39 2011-01-01 16:56:37 #4: 2011-01-01 03:00:00  1 2011-01-01 07:26:39 2011-01-01 16:56:37 #5: 2011-01-01 04:00:00  1 2011-01-01 07:26:39 2011-01-01 16:56:37 #6: 2011-01-01 05:00:00  1 2011-01-01 07:26:39 2011-01-01 16:56:37

check equality: 

all.equal(train.modified, myrez) # [1] "column 'sunrise': attributes: < component “tzone”: 1 string mismatch > mean relative difference: 1.947453e-05"

there minor difference in results, because of time zone miss-specification. should define needed time zone when converting dated.


Comments

Post a Comment

Popular posts from this blog

PHP and MySQL WP -

i trying accomplish 2 things here code. writing php directly wp page using insert php plugin. 1) query mysql database , return results 1 column 4 columns. 2) make each result own hyperlink directs user new wp page runs new query on page. i able query show results , turn results dynamic hyperlink, cannot formatting down. right returns result 1 column. here have: [insert_php] $servername = "localhost"; $username = "login"; //edited privacy $password = "password"; //edited privacy $database = "database"; //edited privacy // create connection $conn = new mysqli($servername, $username, $password, $database); // check connection if ($conn->connect_error) { die("connection failed: " . $conn->connect_error); } echo "connected successfully"; //get results database $result = mysqli_query($conn,"select mine mines order mine"); $all_property = array(); //declare array saving property //showing proper...
Read more

android - InAppBilling registering BroadcastReceiver in AndroidManifest -

i'm implementing inapp billing in android app. works well, however, i'm trying decouple broadcast receiver activity manifest. suggestion in android's trivialdrive sample: // important: dynamically register broadcast messages updated purchases. // register receiver here instead of <receiver> in manifest // because call getpurchases() @ startup, therefore can ignore // broadcasts sent while app isn't running. // note: registering listener in activity bad idea, done here // because sample. regardless, receiver must registered after // iabhelper setup, before first call getpurchases(). currently there's class extends broadcastreceiver : public class iabbroadcastreceiver extends broadcastreceiver { /** * intent action receiver should filter for. */ public static final string action = "com.android.vending.billing.purchases_updated"; private final iabbroadcastlistener mlistener; public iabbroadcastreceiver(iabbroadcastlistener listener) { m...
Read more

go - golang pprof for c library code -

i have been trying profile golang application pprof, seems details of c code(linked static library) missed in output result. flame graph from frame graph can see c code time consuming reported within "runtime._externalcode", no details @ all. , sure static library compiled debug enable. so there way make pprof details(like stack trace) of 3rd party c code? or there other golang profiling tool this?
Read more