Building a SolarEdge Windows Desktop Widget - Part 4 - Finished For Now

The last few days when I wasn't working on actual work, was spent mostly cleaning up and finishing up the work on this project. There were a few major changes between now and then that I want to focus on.

The first big change was to the methods for handling the data points that are different for 2 users.  Username, Password, and SiteId.  Storing all of these is a relatively trivial thing other than Password.  I did not want to store the password in plain text, but was uncertain how to encrypt it in a way that wouldn't be quickly and easily decryptable.  I ended up going with this solution http://weblogs.asp.net/jongalloway//encrypting-passwords-in-a-net-app-config-file. The DPAPI isn't perfect, but it would handle securing the text and tying it to the currently logged in windows user.

I created a simple settings form to handle adding these values in.  In doing this I felt if I was going to spend the time securing the password I should go all the way and make sure the input was secure.  To that end I found http://weblogs.asp.net/pglavich/440191.  This allowed me to ensure that the text in memory from the Textbox wasn't vulnerable to being read by using SecureString.

There is one other setting that I found needed to be added after some time monitoring my solar system; Tolerance Offset.  I have 2 different methods of monitoring my solar generation.  One is through the inverter and thus through the SolareEdge monitoring site.  The other is a revenue grade electrical meter.  After a week or so I noticed that the 2 no longer agreed with each-other on how much I was generating.  After talking to the installer and to SolarEdge, it turns out the inverter has a +/- 5% tolerance for monitoring.  I find this odd, but it is what it is.  The Tolerance Offset will modify all the data by a certain %.  For example, my inverter reports roughly 4.2% more generation than is actually happening.  So I would put in 95.8 in the tolerance offset.  This isn't exactly perfect, but it gets it within a couple kWh for now.  I will probably need to tweak it as the years go on.

The next big change that needed to happen was to gain access to these settings.  To this I added an icon of a gear. Additionally I added a refresh and arrows icon (for dragging).  The icons are from http://www.flaticon.com/.   The refresh and configure icons were fairly straight forward, but the move icon required a little extra work. I used this method to handle it: http://stackoverflow.com/questions/1592876/make-a-borderless-form-movable.  The icons only show themselves when you hover over the right area of the gadget.

The last major addition was a 3rd graph that would show generation by month for the whole year.

Overall this project has been quite enjoyable if not a bit more time consuming than I had originally planned.  There are surely improvements to be made, but for now I need to move onto the next thing.  Probably the biggest improvement would be to give more control over the colors (or rather, any control).  Right now you can have it in any colorset that you want as long as the color is black, dark blue and light blue.

The code exists here https://github.com/iamwyza/SolarEdgeGagdet.  Here are the final images of the design.

Building a SolarEdge Windows Desktop Widget - Part 3

When I first started down this path I had decided that a C# program just wasn't going to be feasible due to the lack of a function to keep it at bottommost (or, in other words, on the desktop).  This in spite of the fact that as far as writing windows programs goes, C# is definitely the language I know the most about.  After the issues I had with the gadgets I decided to take another look at making a C# implementation work.

I started looking around Google to see if I couldn't find something that would work.  I finally settled on the 2nd sub answer of the accepted answer on this stackoverflow http://stackoverflow.com/questions/2027536/setting-a-windows-form-to-be-bottommost.  I also combined that method with a few events that any time the form tried to move forward, it would be sent back.  Essentially this keeps it at the bottom no matter what.

There were a few essential pieces.  In the main class:

[DllImport("user32.dll", EntryPoint = "SetWindowPos")]
public static extern IntPtr SetWindowPos(IntPtr hWnd, int hWndInsertAfter, int x, int Y, int cx, int cy, int wFlags);

Then we need to override the WndProc method. The key here being the SWP_NOZORDER and SWP_NOOWNERZORDER. Basically we don't want it to change the z-index of the form.

        [System.Security.Permissions.PermissionSet(System.Security.Permissions.SecurityAction.Demand, Name = "FullTrust")]
        protected override void WndProc(ref Message m)
        {
            const short SWP_NOZORDER = 0X0004;
            const short SWP_NOACTIVATE = 0X0010;
            const short SWP_NOOWNERZORDER = 0x0200;
            // Listen for operating system messages. 
            switch (m.Msg)
            {
                case WM_WINDOWPOSCHANGING:
                    SetWindowPos(this.Handle, 0, this.Left, this.Top, this.Width, this.Height, SWP_NOACTIVATE | SWP_NOZORDER | SWP_NOOWNERZORDER);

                    // Invalidate to get new text painted. 
                    this.Invalidate();

                    break;

            }
            base.WndProc(ref m);
        }

Once I had taken care of keeping the window on the bottom, the rest of the work was relatively straightforward. This was largely in part due to the fact that I was able to re-use the existing data fetching library I had written for the gadget. (Though I no longer run it in a separate program, it is now handled internally)  At the moment there are 3 main areas of the window.

The first area is the right most area which gives simple current metrics for current power generation in Watts (or Kilowatts usually), then power generated today, this month, this year, and all time (in kWh).  This is all coded with simple labels and set via the .Text property:

lblCurrentPower.Text = data.overviewData.currentPower;
lblCurrentMonth.Text = data.overviewData.lastMonthEnergy + " kWh";
lblLifetimePower.Text = data.overviewData.lifeTimeEnergy + " kWh";
lblTodayPower.Text = data.overviewData.lastDayEnergy + " kWh";
lblLastUpdatedValue.Text = DateTime.Now.ToString();

The second area of the window is the Week view. This is a graph that gives the point in time power generation metrics (in Watts). For this I leveraged the standard windows forms graphing capabilities. Mostly I used the design view to setup the graph, but a few of them I setup in code. One thing I found is that if you put a null after the datapoint you end up with it using whatever the last value was as the next data point. This results in having it show you generating this constant amount of power, when that isn't really the case. To combat this I check for null and do 0 instead if null is found.

            chartWeek.Series[0].XValueType = ChartValueType.DateTime;
            chartWeek.ChartAreas[0].AxisX.LabelStyle.Format = "MM-dd";
            chartWeek.ChartAreas[0].AxisX.Interval = 1;
            chartWeek.ChartAreas[0].AxisX.IntervalType = DateTimeIntervalType.Days;
            chartWeek.ChartAreas[0].AxisX.IntervalOffset = 1;

And when setting the data

            chartWeek.ChartAreas[0].AxisX.Minimum = new DateTime(1970, 1, 1).AddMilliseconds(data.energyChartData.start_week).ToOADate();
            chartWeek.ChartAreas[0].AxisX.Maximum = new DateTime(1970, 1, 1).AddMilliseconds(data.energyChartData.start_week + (data.energyChartData.power_chart_week.Count * 1000 * 60 * 15)).ToOADate();
            chartWeek.Series[0].Points.Clear();

            for (var i = 0; i < data.energyChartData.power_chart_week.Count; i++)
            {
                chartWeek.Series[0].Points.AddXY(new DateTime(1970, 1, 1).AddMilliseconds(data.energyChartData.start_week + (i * 1000 * 60 * 15)), data.energyChartData.power_chart_week[i] == null ? 0 : data.energyChartData.power_chart_week[i]);
            }

I also found and modified some code to do mouse-over tooltips (link). The modification primarily stems from needing to make sure the tooltip wasn't getting re-drawn every time the mouse moved. Normally this isn't a huge issue, but I also customized the tooltip class and when it would redraw you'd end up with this annoying flicker.

void chartmonth_MouseMove(object sender, MouseEventArgs e)
        {
            var pos = e.Location;
            if (prevPosition.HasValue && pos == prevPosition.Value)
                return;

            prevPosition = pos;
            var results = chartMonth.HitTest(pos.X, pos.Y, false,
                                            ChartElementType.DataPoint);
            if (results.Any())
            {
                
                foreach (var result in results)
                {
                    if (result.ChartElementType == ChartElementType.DataPoint)
                    {
                        var prop = result.Object as DataPoint;
                        if (prop != null)
                        {
                            var pointXPixel = result.ChartArea.AxisX.ValueToPixelPosition(prop.XValue);
                            var pointYPixel = result.ChartArea.AxisY.ValueToPixelPosition(prop.YValues[0]);

                            // check if the cursor is really close to the point (2 pixels around the point)
                            if (Math.Abs(pos.X - pointXPixel) < 30 &&
                                Math.Abs(pos.Y - pointYPixel) < 15)
                            {
                                
                                if ((double)tooltip.Tag != prop.YValues[0])
                                {
                                    tooltip.Show(prop.YValues[0] + " kWh", chartMonth,
                                                    pos.X + 15, pos.Y - 15);
                                    tooltip.Tag = prop.YValues[0];
                                }
                            }
                            else
                            {
                                tooltip.RemoveAll();
                                tooltip.Tag = 0.0;
                            }
                        }
                    }else
                    {
                        tooltip.RemoveAll();
                        tooltip.Tag = 0.0;
                    }
                }
            }
            else
            {
                tooltip.RemoveAll();
                tooltip.Tag = 0.0;
            }
        }

The tooltip class looks like this. It was based off of this post, but with vastly better color choices :).

    class CustomToolTip : ToolTip
    {
        public CustomToolTip()
        {
            this.OwnerDraw = true;
            this.Popup += new PopupEventHandler(this.OnPopup);
            this.Draw += new DrawToolTipEventHandler(this.OnDraw);
        }

        private void OnPopup(object sender, PopupEventArgs e) // use this event to set the size of the tool tip
        {
            if ((double)Tag >= 100)
                e.ToolTipSize = new Size(100, 25);
            else
                e.ToolTipSize = new Size(80,25);
            
        }

        private void OnDraw(object sender, DrawToolTipEventArgs e) // use this event to customise the tool tip
        {
            Graphics g = e.Graphics;

            g.FillRectangle(Brushes.Black, e.Bounds);

            g.DrawRectangle(new Pen(new SolidBrush(Color.FromArgb(0, 131, 255)), 1), new Rectangle(e.Bounds.X, e.Bounds.Y,
                e.Bounds.Width - 1, e.Bounds.Height - 1));

            g.DrawString(e.ToolTipText, new Font(e.Font, FontStyle.Bold), Brushes.White, new PointF(e.Bounds.X + 5, e.Bounds.Y + 5)); // top layer
        }
    }

The third and final current piece of the form is the month view. This gives you the daily generation of power (in kWh) for the current month. This uses a standard bar graph and also almost identical tooltips. I don't know why, but the first data point in a bar graph is cut in half. To alter this I changed the minimum of the graph to .5 and offset it by .5.

            chartMonth.ChartAreas[0].AxisX.Minimum = .5;
            chartMonth.ChartAreas[0].AxisX.IntervalOffset = .5;
            chartMonth.ChartAreas[0].AxisX.Interval = 1;
            lblMonthNow.Text = DateTime.Now.ToString("MMMM");

Populating the graph was very similar to the week code:

            chartMonth.Series[0].Points.Clear();

            for (var i = 0; i < data.energyChartData.energy_chart_month_by_day.production.data.Count; i++)
            {
                chartMonth.Series[0].Points.AddXY(i+1, data.energyChartData.energy_chart_month_by_day.production.data[i]);
            }

When all is said and done the results look something like this:

So what's left now? The window is currently hard coded to sit on a specific place on my desktop.  This works, but I would much rather be able to set it dynamically with a simple drag.  The username, password, and site ID for the connection back to solaredge is also hard coded.  Both of these need to be moved into a configuration file for persistence and to reduce the effort to set up the interface.  I'm not 100% happy on the aesthetic designs yet either, but haven't settled on what changes I'd like to make to improve it.  There are also some places in the code that needs cleaning and simplifying.  Lastly I still am considering adding the year view to this as well.  Once I'm happy with the full design I will release the code for anyone to use.

Building a SolarEdge Windows Desktop Widget - Part 2

Part 1: http://iamwyza.blogspot.com/2015/04/building-solaredge-windows-desktop.html

Onwards and upwards we go.

The next step in the adventure was to start working on the gadget code itself.  Chrome is very frustrating to deal with when you are doing localhost ajax calls.  There are ways to do it, but they all involve adding a little risk to your own security. Since I use chrome for everything else, I figured I'd suck it up and do it in IE.  After all Windows Gadgets are rendered with the IE engine. (though I'll later find out it is the IE 7 engine, *shutter*).

Whenever I'm dealing with web related stuff I almost always pull in at least 1 javascript library.  Realistically there isn't a good reason not to.  In my case it is nearly always jQuery.  I'm familiar with it and it is has been quite stable.  The other library I used was Flot for the graphing functions.  I also needed excanvas in order to provide HTML 5's canvas support to IE7 (which Flot needs).


So the first area to look at is the basic setup of the javascript.  We have a few variables here are going to be used with the scope of this page.  We next have a function defined "Update()" which we'll use any time we want to request new data from the server and update the various fields and plots.

var first = true;

// Last 6 days of power setup:
jQuery.support.cors = true;
$(function(){
 var previousPoint = null, previousLabel = null;
 var powerpoints = [];
 var days = [];
 var daysrendered = false;
 var monthsrendered = false;
    
 //Here we fetch the data and render it
 function Update() {
  $.ajax({ 
   url: "http://localhost:8001", 
   type: "GET", 
   dataType:"json", 
   success: function(data) { 
    //Set the simple values
    $('#currentPower').html(data.overviewData.currentPower);
    $('#powerToday').html(data.overviewData.lastDayEnergy);
    $('#powerMonth').html(data.overviewData.lastMonthEnergy);
    $('#powerTotal').html(data.overviewData.lifeTimeEnergy);
    
    //Setup the power production per point of time for the last week
    $.each(data.energyChartData.power_chart_week, function(i, e) {
     powerpoints.push([data.energyChartData.start_week+(i*1000*60*15), e]);
    });
    
    //Month long daily power production
    $.each(data.energyChartData.energy_chart_month_by_day.production.data, function(i,e){
     days.push([i,e]);
    });
    
    //Determine which plot was visiable.  We need to know this because rendering a plot when you can't see it will cause issues.
    if ($('#previousDays').is(':visible')){
     renderDays();
     monthsrendered = false;
    }else{
     renderMonths();
     daysrendered = false;
    }
    
   // Show current datetime to make sure it is still running
   var currentdate = new Date(); 
   $('#lastUpdated').html(currentdate.getDate() + "/"
   + (currentdate.getMonth()+1)  + "/" 
   + currentdate.getFullYear() + " "  
   + currentdate.getHours() + ":"  
   + currentdate.getMinutes() + ":" 
   + currentdate.getSeconds());
   
   setTimeout(function() { Update(); }, 300000);
   }
   
  });
 }

The following 2 functions are in charge of setting up the plots themselves.  Each plot serves a different purpose.  The "days" plot is a line plot that shows how much power is being produced at that point in time.  It has data points in 15 minute intervals.   The "month" plot shows how much power was generated each day in the current month.

 //Renders the plot for the last week's power production over time (kW)
 function renderDays() {
  $.plot('#previousDays', [powerpoints], { 
    xaxis: { mode: "time",
      axisLabel: "Day"
      },
    yaxis: {
       tickFormatter: function (v, axis) {
        return v + " kW";
       },
       axisLabel: "kW"
    },
    grid: {
     hoverable: true,
     borderWidth: 0
    },
    colors: ["#5DFF00"],
    series: {
     lines:{
      fill:true,
      fillColor: {colors: [{opacity: 0.1}, {opacity: 0.7}]}
     }
    }
   });
   $("#previousDays").UseTooltip();
   daysrendered = true;
 }

 //renders the plot for the current months power production per day (kWh)
 function renderMonths() {
  $.plot('#month', [days], {
   series:{
    bars:{
     show: true
    }
   },
   yaxis: {
      tickFormatter: function (v, axis) {
       return v + " kWh";
      },
      axisLabel: "kWh",
      axisLabelPadding: 10
   },
   grid: {
    hoverable: true,
    borderWidth: 0
   },
   colors: ["#5DFF00"]
  });
  
  
  $("#month").UseTooltip();
  monthsrendered = true;
 }  

Next we bind the useToolTip function to our own functionality.  We want to show the tooltip up and over a little with  the same colors that the graph uses.  It also determines which graph we are on and uses the appropriate unit of measure.

   
 //Describes and sets up the tooltip display
 $.fn.UseTooltip = function () {
  $(this).bind("plothover", function (event, pos, item) {
   if (item) {
    if ((previousLabel != item.series.label) || (previousPoint != item.dataIndex)) {
     previousPoint = item.dataIndex;
     previousLabel = item.series.label;
     $("#tooltip").remove();

     var x = item.datapoint[0];
     var y = item.datapoint[1];

     var color = item.series.color;

     if ($('#previousDays').is(':visible')){
     
      showTooltip(item.pageX,
      item.pageY,
      color,
      "" + y + " kW");
     }else{
      showBarTooltip(item.pageX,
      item.pageY,
      color,
      "" + y + " kWh");
     }
    }
   } else {
    $("#tooltip").remove();
    previousPoint = null;
   }
  });
 };
 


 
 //Show Tooltip on hover
 function showTooltip(x, y, color, contents) {
  $('' + contents + '').css({
   position: 'absolute',
   display: 'none',
   top: y - 20,
   left: x - 120,
   border: '2px solid ' + color,
   padding: '3px',
   'font-size': '9px',
   'border-radius': '5px',
   'background-color': '#000000',
   'font-family': 'Verdana, Arial, Helvetica, Tahoma, sans-serif',
   opacity: 0.9
  }).appendTo("body").fadeIn(200);
 }
 
 //Show Tooltip on hover
 function showBarTooltip(x, y, color, contents) {
  $('' + contents + '').css({
   position: 'absolute',
   display: 'none',
   top: y + 20,
   left: x + 60,
   border: '2px solid ' + color,
   padding: '3px',
   'font-size': '9px',
   'border-radius': '5px',
   'background-color': '#000000',
   'font-family': 'Verdana, Arial, Helvetica, Tahoma, sans-serif',
   opacity: 0.9
  }).appendTo("body").fadeIn(200);
 }

Lastly we have the click events on a couple buttons to switch between week and month graphs. Also we need to run Update() for the first time.

 //Normally I'd use jQuery.on('click') here, but IE 7 + Gadgets doesn't work for that.  So back in time we go.
 document.getElementById("weakLink").onclick = function(){
  $('#previousDays').show();
  $('#month').hide();
  if (! daysrendered){
   renderDays();
  }
 };
 document.getElementById("monthLink").onclick = function(){
  $('#previousDays').hide();
  $('#month').show();
  if (! monthsrendered) {
   renderMonths();
  }
 };  

 Update();
 });

A little bit of CSS.

* {
 color: #e4e4e4;
}

.currentSubDiv {
 border-top: 1px solid #0083FF;
 padding-top:5px; 
 margin-top:5px;
 margin-left:5px;
}
.currentSubDivValue { 
 color: #5DFF00;
 font-style:bolder;
}
.previousDayDiv {
 width: 1000px;
 height:325px;
 float:left;clear:left;
}

And finally some basic and mostly unpolished html.

<body style="background-color:#000000; width:1200px; height:400px;">
<div style="width:1175px; height: 375px; border: 1px solid #000000">
 <div style="float:left;text-align:center;">
  <div style="border-bottom: 1px solid #0083FF;float:left;clear:left;">
   <div style="width:495px; cursor:pointer; float:left;" id="weakLink">Week</div>
   <div style="float:left">|</div>
   <div style="width:495px; cursor:pointer; float:left;" id="monthLink">Month</div>
  </div>
  <div class="previousDayDiv" id="previousDays"></div>
  <div class="previousDayDiv" id="month" style="display:none;"></div>
 </div>
 <div style="float:right;border-left:1px solid #0083FF; width:125px;height: 375px; text-align:center;font-size:22px;">
  <div>
   Current
  </div>
  <div class="currentSubDiv">
   Power<br />
   <span id="currentPower" class="currentSubDivValue">N/A</span>
  </div>
  <div class="currentSubDiv">
   Today<br />
   <span id="powerToday" class="currentSubDivValue">N/A</span>
  </div>
  <div class="currentSubDiv">
   Month<br />
   <span id="powerMonth" class="currentSubDivValue">N/A</span>
  </div>
  <div class="currentSubDiv">
   Lifetime<br />
   <span id="powerTotal" class="currentSubDivValue">N/A</span>
  </div>
  <div class="currentSubDiv">
   Last Updated<br />
   <span id="lastUpdated" class="currentSubDivValue">N/A</span>
  </div>
 </div>
</div>
</body>
</html>

So there we have it.  All the code to make this stuff work right in theory.  When it renders in IE, life is good and it looks like this:

Next step was to get it working as a gadget.  I setup the necessary files (you can google the layout) and dropped it in.  I then found that I needed an older version of jQuery to support IE7.  Fixed that and I had a gadget.  However over the last few days I've had issues with the gadget not refreshing itself every 5 minutes as planned.  Over time it slows down and eventually crashes. Doing research on this problem, it is apparently common, even with the default gadgets, to suck up memory due to leaks and poor design in the engine.  The only fix anyone has really given for it is to restart gadgets with a batch file or vbs file.  I attempted that route, but when it restarted the widget wouldn't come back (in fact no widgets worked). The only way to get them to come back was to reboot.

So almost all of the work in this post has been for naught.  However I do have a new direction I started taking yesterday and today (when I wasn't feeling crappy).  That'll be for the next post.

Building a SolarEdge Windows Desktop Widget - Part 1

Let me preface this with saying alot of Googling was done to accomplish the task. Some of the code below (especially the listener area) was pieced together from posts others have made.

Recently I had solar panels installed on my home.  There are tomes of information written about the subject of solar.  The fact is that for my area, in 8 years I'll break even and then I have 17-25 years of the majority of my electrical generation take care of.  But that isn't what this post is about.

SolarEdge, the company that produces the Power Optimizers and Inverter, provides a pretty nice web interface to monitor a whole slew of data down to individual panel power generation. (link)  That's great, I love it, but I already have a bunch of graphs and such on my desktop that tracks various metrics for my desktop machine. This is mostly provided via Aquaero + Aquasuite by AquaComputer.  I also use Rainmeter for a few things like disk and network utilization.

I figured adding some near real time data from the solar install to the desktop was a natural extension of my obsession with data.  My first inclination was to approach it from trying to see if I could access the data directly from the inverter itself.  I fiddle around with nmap and packet sniffing and was able to isolate the data transmission from the inverter up to the server.  Unfortunately it is either encrypted or at the very least encoded somehow.  Now I could spend a bunch of time reverse-engineering it, but the reality is I don't *need* real time data, just near real time.  Based on my observations of the tracking site and the packet sniffing, it appears that the data is shipped over every minute and then summarized every 15 minutes.  So the site would be a good place to get this information if it is in a relatively easy format to parse.

Firing up Chrome's developer tools I checked out what was being sent to the browser.  Looking through the various calls I found this: https://monitoring.solaredge.com/solaredge-web/p/public_dashboard_data?fieldId=111098 (well, its slightly different for the logged in version, but close enough).  Going here gives me a nice pretty JSON full of data to use.

{
"sumData":{
            "siteName":"Wyza",
  
  "siteInstallationDate":"04/02/2015",
  "sitePeakPower":"11.34",
              "siteAddress":"",
        "siteCountry":"United States",
  "siteState":"Indiana",  
    "siteUpdateDate":"04/07/2015 20:21",
      "maxSeverity":"0",
    "status":"Active"
},
"instData":{
  "installerImage":"Morton Solar Logo 1.jpg",
  "installerImagePath":"/installerImage?fieldId=123456",
  "installerImageHash":"565362791",
  useDefault:true
},
"imgData":{
  "fieldId":"123456",
  "image":"house.jpg",
  "imagePath":"/fieldImage?fieldId=123456",
  "imageHash":"277481497"
},
"overviewData":{
    "lastDayEnergy":"21.82 kWh",
      "lastMonthEnergy":"214.35 kWh",
      "currentPower":"0 W",
      "lifeTimeEnergy":"215.24 kWh"
            ,currencyCode:'USD'
      ,revenue:'35.16975'
      ,formatedRevenue:'$35.17'
  },
"savingData":{
    "CO2EmissionSaved":"326.91 lb",
    "treesEquivalentSaved":"8.39",
  "powerSavedInLightBulbs":"652.23"
},
 powerChartData:
 {
  "start_week":1427846400000,
  "energy_chart_month":[
  {"name":2015,"data":[0,0,0,214.349,0,0,0,0,0,0,0,0]}
  ],
  "energy_chart_month_max":214.349,
  "month_uom":"kilo",
  "energy_chart_quarter":[
  {"name":2015,"data":[0,214.349,0,0]}
  ],
  "quarter_uom":"kilo",
  "year_categories":[2015],
  "energy_chart_year":[
  {"name":2015,"data":[214.349]}
  ],
  "year_uom":"kilo"
 },
 energyChartData:
 {
     "field_start_date":{"year":2015,"month":3,"day":1},
   "field_end_date":{"year":2015,"month":3,"day":7},
      "year_range":[[2015]],
    start_week:1427846400000,
GMTOffset:new Date(1427846400000).getTimezoneOffset()*60000,
end_week:{"year":2015,"month":3,"day":7},
end_week_next:1428451140000,
power_chart_week:[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,4.365,4.343,1.238,1.256,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,1.541,1.612,null,1.549,1.272,0.674,0.74,0.827,0.67,1.154,1.533,1.184,0.971,2.507,8.415,7.303,6.05,4.905,1.791,1.965,2.847,4.393,6.789,7.169,6.385,3.455,5.754,6.331,5.411,2.899,2.95,0.776,0.185,0.042,0.354,1.63,1.166,0.311,0.015,0,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0.019,0.098,0.122,0.134,0.184,0.001,0.011,0.005,0.005,0,0.031,0.108,0.278,1.098,3.151,4.41,4.325,3.738,3.977,3.522,2.117,0.589,0.472,0.12,0.313,0.567,0.579,2.35,1.748,1.045,0.579,1.479,0.555,0.734,1.095,0.912,1.2,1.416,1.087,0.826,0.657,0.509,0.449,0.588,0.668,0.575,0.373,0.262,0.055,0,0,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,0.063,0.376,0.776,1.241,1.8,2.468,3.066,3.668,4.286,5.005,5.637,6.141,6.511,6.961,7.334,7.662,8.046,8.346,8.686,9.132,9.152,9.082,9.435,9.42,9.487,9.409,9.16,9.04,9.13,9.024,8.915,8.62,8.501,8.208,7.726,7.36,7.001,6.571,6.117,5.634,5.093,4.582,4.044,3.443,2.84,2.267,1.682,1.207,0.755,0.385,0.047,0,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,0.06,0.424,0.786,1.428,2.082,3.01,3.552,2.813,2.197,4.68,4.767,5.177,5.542,6.977,7.008,7.55,8.097,8.759,8.379,8.9,8.642,8.574,8.577,7.476,8.035,6.327,6.769,5.678,5.935,7.512,6.351,5.577,4.938,5.862,4.383,4.004,3.382,2.967,2.663,2.467,2.145,2.038,2.022,1.797,1.267,1.07,0.69,0.534,0.297,0.171,0.007,0,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,0,0.146,0.148,0.308,0.326,0.505,0.416,0.491,0.615,0.797,1.101,0.987,0.822,1.224,1.108,2.617,2.912,4.292,4.527,2.993,1.237,0.824,1.961,3.207,2.754,2.621,2.482,3.352,3.102,2.86,1.692,1.034,1.082,2.486,1.008,1.072,1.13,1.213,1.287,2.003,1.799,1.587,0.995,0.799,0.481,0.29,0.415,0.352,0.15,0.014,0,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,0,0,0.168,0.171,0.733,0.811,1.214,3.071,2.887,2.255,3.125,4.192,4.068,6.575,5.564,3.95,2.796,3.44,4.236,4.404,2.721,3.026,2.031,2.125,2.036,2.342,4.584,4.414,3.426,1.625,0.81,0.292,null,null,null,null,null,0,0.706,0.153,0.08,0.014,0.027,0.163,0.305,0.204,0.3,0.443,0.438,0.134,0.006,0,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],
power_chart_week_c:[],
power_chart_week_sc:[],
"energy_chart_month_by_day":{
"production":{"total":"214.349 kWh","roundValueTotal":"214.3489990234375","data":[2.345,26.07,14.175,74.674,56.923,18.344,21.818,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]}
},
"energy_chart_month_by_day_total":214.3489990234375,
"energy_chart_month_by_day_max":74.674,
"energy_chart_month_by_day_total_c":0,
"energy_chart_month_by_day_max_c":0,
"energy_chart_month_by_day_total_sc":0,
"energy_chart_month_by_day_max_sc":0,
"energy_chart_year_by_month":{
"production":{"total":"214.349 kWh","roundValueTotal":"214.3489990234375","data":[0,0,0,214.349,0,0,0,0,0,0,0,0]}
},
"energy_chart_year_by_month_total":214.3489990234375,
"energy_chart_year_by_month_max":214.349,
"energy_chart_year_by_month_total_c":0,
"energy_chart_year_by_month_max_c":0,
"energy_chart_year_by_month_total_sc":0,
"energy_chart_year_by_month_max_sc":0,
"prev_month_date":1427899550000,
"next_month_date":1428451199000,
"prev_year_date":1427899550000,
"next_year_date":1428451199000,
"year_uom":"kilo",
"month_uom":"kilo"   },
 performanceRatioData:
 { 
  fieldHasPR:false
   } 
}

So yay, this is good, lots of tasty data to use.  Well almost.  The JSON does not properly conform to JSON standards.  Some of the field keys aren't surrounded by quotes, some of them have single quotes, but the worst part was this line:

GMTOffset:new Date(1427846400000).getTimezoneOffset()*60000,

JSON parsers really don't like this by default.  But at this point I wasn't really sure if that was going to be a problem or not.  My next thought was going to be how to take this data and render it to the desktop.  (and then after determining that I could see how much the JSON formatting was a problem).  Initially I had the thought to use Rainmeter.  I was already using it for other collections and this seemed like a good fit.  I dug into the plugin development area of rainmeter and for a while it seemed like I could make it work.  But I found the biggest problem is that in the graph area it really didn't have a good way to provide historical data, which I really wanted to show. It mostly gave a method to update the graph every x seconds with a new point of data, but no way to pre-populate said graph.  So that line of attack was a bust.

My next line of thinking was to write a Windows application that would stick to the desktop and provide the data.  This does have some promise, but some folks have issue keeping it stuck at the bottom where it belongs.  After mucking around for an hour or so, I really wasn't making progress and the day was wearing on.

My final line of thinking is where I ended up, which is to leverage Windows gadgets.  Gadgets seemed fairly easy to code since they are basically HTML.  The biggest downside here was that the renderer used is IE7. (Seriously MS?  Would it have killed you to patch in the option to use at least IE10? I mean IE is terrible, but 10+ was less so.)  There were other issues with using gagdets, chiefly that the data coming from JSON meant I was doing a cross domain request.  (from localhost to monitoring.solaredge.com).  Now you can get around this with jsonp *mostly*.  But to get the data from being logged in, I needed to make a json request and then make another request with the same cookies and this is somewhat more difficult.  The simple fact is that these constraints with cross domain AJAX requests are there for a reason and are not designed to be easily circumnavigated.   The other problem was that the JSON was not quite what the browser would like (see above).

So the first thing I'd need to code is a proxy service.  This will run locally on the machine and make the requests to the server for the data, clean it up so that it conforms, and then spit it back out on request.  To do that I wrote a C# Windows Form that hides itself on startup and sits in the taskbar.   There were 3 main classes that would handle this work.   First I cleaned up the JSON data manually and used this tool to generate a C# object tree: http://json2csharp.com/. This resulted in the following class:

using System.Collections.Generic;

namespace SolarEdgeParser
{
    public class SumData
    {
        public string siteName { get; set; }
        public string siteInstallationDate { get; set; }
        public string sitePeakPower { get; set; }
        public string siteAddress { get; set; }
        public string siteCountry { get; set; }
        public string siteState { get; set; }
        public string siteUpdateDate { get; set; }
        public string maxSeverity { get; set; }
        public string status { get; set; }
    }

    public class InstData
    {
        public string installerImage { get; set; }
        public string installerImagePath { get; set; }
        public string installerImageHash { get; set; }
        public bool useDefault { get; set; }
    }

    public class ImgData
    {
        public string fieldId { get; set; }
        public string image { get; set; }
        public string imagePath { get; set; }
        public string imageHash { get; set; }
    }

    public class OverviewData
    {
        public string lastDayEnergy { get; set; }
        public string lastMonthEnergy { get; set; }
        public string currentPower { get; set; }
        public string lifeTimeEnergy { get; set; }
        public string currencyCode { get; set; }
        public string revenue { get; set; }
        public string formatedRevenue { get; set; }
    }

    public class SavingData
    {
        public string CO2EmissionSaved { get; set; }
        public string treesEquivalentSaved { get; set; }
        public string powerSavedInLightBulbs { get; set; }
    }

    public class EnergyChartMonth
    {
        public int name { get; set; }
        public List<double> data { get; set; }
    }

    public class EnergyChartQuarter
    {
        public int name { get; set; }
        public List<double> data { get; set; }
    }

    public class EnergyChartYear
    {
        public int name { get; set; }
        public List<double> data { get; set; }
    }

    public class PowerChartData
    {
        public long start_week { get; set; }
        public List<EnergyChartMonth> energy_chart_month { get; set; }
        public double energy_chart_month_max { get; set; }
        public string month_uom { get; set; }
        public List<EnergyChartQuarter> energy_chart_quarter { get; set; }
        public string quarter_uom { get; set; }
        public List<int> year_categories { get; set; }
        public List<EnergyChartYear> energy_chart_year { get; set; }
        public string year_uom { get; set; }
    }

    public class FieldStartDate
    {
        public int year { get; set; }
        public int month { get; set; }
        public int day { get; set; }
    }

    public class FieldEndDate
    {
        public int year { get; set; }
        public int month { get; set; }
        public int day { get; set; }
    }

    public class EndWeek
    {
        public int year { get; set; }
        public int month { get; set; }
        public int day { get; set; }
    }

    public class Production
    {
        public string total { get; set; }
        public string roundValueTotal { get; set; }
        public List<double> data { get; set; }
    }

    public class EnergyChartMonthByDay
    {
        public Production production { get; set; }
    }

    public class Production2
    {
        public string total { get; set; }
        public string roundValueTotal { get; set; }
        public List<double> data { get; set; }
    }

    public class EnergyChartYearByMonth
    {
        public Production2 production { get; set; }
    }

    public class EnergyChartData
    {
        public FieldStartDate field_start_date { get; set; }
        public FieldEndDate field_end_date { get; set; }
        public List<List<int>> year_range { get; set; }
        public long start_week { get; set; }
        public string GMTOffset { get; set; }
        public EndWeek end_week { get; set; }
        public long end_week_next { get; set; }
        public List<double?> power_chart_week { get; set; }
        public List<object> power_chart_week_c { get; set; }
        public List<object> power_chart_week_sc { get; set; }
        public EnergyChartMonthByDay energy_chart_month_by_day { get; set; }
        public double energy_chart_month_by_day_total { get; set; }
        public double energy_chart_month_by_day_max { get; set; }
        public int energy_chart_month_by_day_total_c { get; set; }
        public int energy_chart_month_by_day_max_c { get; set; }
        public int energy_chart_month_by_day_total_sc { get; set; }
        public int energy_chart_month_by_day_max_sc { get; set; }
        public EnergyChartYearByMonth energy_chart_year_by_month { get; set; }
        public double energy_chart_year_by_month_total { get; set; }
        public double energy_chart_year_by_month_max { get; set; }
        public int energy_chart_year_by_month_total_c { get; set; }
        public int energy_chart_year_by_month_max_c { get; set; }
        public int energy_chart_year_by_month_total_sc { get; set; }
        public int energy_chart_year_by_month_max_sc { get; set; }
        public long prev_month_date { get; set; }
        public long next_month_date { get; set; }
        public long prev_year_date { get; set; }
        public long next_year_date { get; set; }
        public string year_uom { get; set; }
        public string month_uom { get; set; }
    }

    public class PerformanceRatioData
    {
        public bool fieldHasPR { get; set; }
    }

    public class SolarEdgeData
    {
        public SumData sumData { get; set; }
        public InstData instData { get; set; }
        public ImgData imgData { get; set; }
        public OverviewData overviewData { get; set; }
        public SavingData savingData { get; set; }
        public PowerChartData powerChartData { get; set; }
        public EnergyChartData energyChartData { get; set; }
        public PerformanceRatioData performanceRatioData { get; set; }
    }
}

Next we need to do the work to fetch the data, deserialize it into a POCO (Plain Old C# Object), and then re-serialize it. I do this so that it will return a nice clean well formatted JSON. To fetch it we need to first login to get a session cookie. Then we'll use that to go fetch the data.

using System;
using System.Net;
using System.Text.RegularExpressions;
using System.IO;
using System.Web.Script.Serialization;

namespace SolarEdgeParser
{
    public class SolarEdge
    {
        public string GetSerializedData()
        {
            var cookiejar = new CookieContainer();
            Login(ref cookiejar);

            var request = (HttpWebRequest)WebRequest.Create("https://monitoring.solaredge.com/solaredge-web/p/dashboard_data?fieldId=111098");
            request.CookieContainer = cookiejar;

            var response = (HttpWebResponse)request.GetResponse();
            var data = response.GetResponseStream();
            var reader = new StreamReader(data);
            var result = reader.ReadToEnd();

            result = Regex.Replace(result, @"GMTOffset:new Date\(\d+\)\.getTimezoneOffset\(\)\*60000", "GMTOffset: " + TimeZoneInfo.Local.GetUtcOffset(DateTime.Now).Hours);
            var parsedData = new JavaScriptSerializer().Deserialize<SolarEdgeData>(result);

            reader.Close();
            data.Close();
            response.Close();
            return new JavaScriptSerializer().Serialize(parsedData);
        }

        void Login(ref CookieContainer cookiejar)
        {
            var request = (HttpWebRequest)WebRequest.Create("https://monitoring.solaredge.com/solaredge-web/p/submitLogin");
            request.CookieContainer = cookiejar;
            request.Method = "POST";
            request.ContentType = "application/x-www-form-urlencoded; charset=UTF-8";
            using (var writer = new StreamWriter(request.GetRequestStream()))
            {
                writer.Write("cmd=login&demo=false&username=nope&password=notachance");
                writer.Flush();
            }

            var response = (HttpWebResponse)request.GetResponse();
            var data = response.GetResponseStream();
            var reader = new StreamReader(data);
            var result = reader.ReadToEnd();
            reader.Close();
            data.Close();
            response.Close();
            if (Regex.Match(result, "Failure", RegexOptions.IgnoreCase).Success)
            {
                throw new Exception("Login Failure");
            }
        }
    }
}

Now to access this data I decided to use a HttpListener. It is a very lightweight method to interact over http. Since I'm pulling the data via AJAX this is the obvious and simplest solution.

using System;
using System.Windows.Forms;
using System.Net;
using SolarEdgeParser;

namespace SolarEdgeProxy
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
            notifyIcon1.Visible = true;
            var service = new SolarEdgeProxyService();
            this.Hide();
        }


        public class SolarEdgeProxyService
        {
            private Listener listener;
            public SolarEdgeProxyService()
            {
                listener = new HttpListener();
                listener.Prefixes.Add("http://localhost:8001/");
                listener.Start();
                IAsyncResult result = listener.BeginGetContext(new AsyncCallback(ListenerCallback), listener);
            }

            public static void ListenerCallback(IAsyncResult result)
            {
                HttpListener listener = (HttpListener)result.AsyncState;
                // Call EndGetContext to complete the asynchronous operation.
                HttpListenerContext context = listener.EndGetContext(result);
                HttpListenerRequest request = context.Request;
                // Obtain a response object.
                HttpListenerResponse response = context.Response;
                // Construct a response. 
                var se = new SolarEdge();
                var responseString = se.GetSerializedData();
                byte[] buffer = System.Text.Encoding.UTF8.GetBytes(responseString);
                // Get a response stream and write the response to it.
                response.ContentLength64 = buffer.Length;
                System.IO.Stream output = response.OutputStream;
                output.Write(buffer, 0, buffer.Length);
                // You must close the output stream.
                output.Close();
                listener.BeginGetContext(new AsyncCallback(ListenerCallback), listener);
            }
        }

        private void Form1_Resize(object sender, EventArgs e)
        {
            
            this.Hide();
        }

        private void notifyIcon1_MouseDoubleClick(object sender, MouseEventArgs e)
        {
            this.Show();

        }
    }
}

So there we go. Now I have a service listening on localhost:8001 that will handle all the work of getting me my data. Now it is time to turn to developing the gadget itself. But that post is for another day.