Deleting old files by using schedule task and PowerShell

If you are administrator or programmer, you probably were in situation that you wanted to free up some server space by deleting old unused files. Following script may be quite useful if you want to get rid of old files of the certain type – also if they are packed into .zip packages.

Script checks for all predefined file types with the creation date older than 6 months (can be configured). If an old file is detected then it is being deleted and that information is being stored to the local log.txt file. The old .zip files will also be deleted (containing only the files we want to delete).

The recommended way of using it is to create windows schedule task that runs periodically, so the server space usage is always optimized.

delete_old_files

 param (
[int]$TestMode = 1,
[int]$DeleteOlderThanMonths = 6,
[string]$DeleteFileExtension = ".jpg",
[string]$Root_Folder = "C:\",
[string]$Logfile = "Log.txt"
)

#writes log messages
Function LogWrite
{
   Param ([string]$logstring)
   $currentdate = get-date -format "M-d-yyyy HH:MM"
   Add-content $Logfile -value "$currentdate | $logstring"
}

#checks if zip file contains only DeleteFileExtension file type
Function CanZIPbeDeleted($zipFilePath)
{
  [int]$canBeDeleted = 1;

  try { 
      $zipArchive = Get-Item -Path $zipFilePath | ForEach-Object { 
         (New-Object -ComObject Shell.Application).NameSpace($_.FullName)
      }
  
      foreach($item in $zipArchive.items())
      {
        if(!$item.Path.EndsWith($DeleteFileExtension))
        {
           $canBeDeleted = 0
           $msg = "$zipFilePath cannot be deleted as it contains non $DeleteFileExtension file"
           LogWrite $msg
           Write-Host $msg -foreground Green 
           return 0
        }
     }   
   } 
   catch { 
       $msg = "Unexpected Error. Error details: $_.Exception.Message"
       LogWrite $msg
       Write-Host $msg -foreground Red 
       return 0
   } 
 
    return $canBeDeleted;
}

#deletes the file
Function DeleteFile($filePathToDelete)
{
  try
  { 
     if($TestMode -eq 0)
     {
       Remove-Item $filePathToDelete
     }
     
     $size = (Get-Item -Path $filePathToDelete | Select-Object Name, @{Name="Size";Expression={[string]::Format("{0:0.00} MB", $_.Length / 1MB)}}).Size
     $msg = "File deleted ($size): $filePathToDelete"
     LogWrite $msg 
     Write-Host $msg -foreground "magenta"  
  } 
  catch { 
     $msg = "Unexpected Error. Error details: $_.Exception.Message"
     LogWrite $msg 
     Write-Host $msg -foreground Red 
   }    
}

 
 get-childitem -Path $Root_Folder –recurse |  
 where-object { 
    $_.CreationTime -lt (get-date).addMonths(-$DeleteOlderThanMonths) -and ($_.FullName.EndsWith($DeleteFileExtension) -or $_.FullName.EndsWith(".zip"))
  } | 
 Foreach-Object {
 
    if ($_.FullName.EndsWith(".zip"))
    {
       $canDeleteZip = CanZIPbeDeleted $_.FullName
       if($canDeleteZip -eq 1)
       {
          DeleteFile $_.FullName
       }
    }
    else
    {
       DeleteFile $_.FullName
    }
 } 

Enjoy!
delete_old_files

1 Star2 Stars3 Stars4 Stars5 Stars (1 votes, average: 4.00 out of 5)
Loading...Loading...

SVN log parsing using PowerShell

This script can be useful when gathering SVN release changes that occurred since the last release. It connects to SVN server, gets logs from within date range (last build and date now). Logs are then filtered, cleaned (removed if no comments were added). Next, email message is sent based on the retrieved data. To be able to run the script, you need to configure it by setting-up your own svn server url, credentials, also please make sure you have enabled running ps scripts on your server.

Enjoy!

svn_log_parser

 param (
[string]$Version = "2.12.0.2",
[string]$SendEmailsTo = "test@test.com",
[string]$SVNUser = "user",
[string]$SVNPass = "pass",
[string]$SVNbranchRoot = "https://svn.test.com/www/Branches/MyApp/2.12.0"
)

#get svn logs for the current release
 Function GetSVNLogs($lastBuildDate_)
 {
    [string]$fromDate = $lastBuildDate_.ToString("yyyy-MM-dd");
    [string]$toDate = (get-date).ToString("yyyy-MM-dd");
    [string]$RevisionDates = "{$fromDate}:{$toDate}"; 

    #add -v param for verbose
    1$log = 1(&$svn log $SVNbranchRoot -r $RevisionDates --limit 500 --xml --username $SVNUser --password $SVNPass);

    $logObjects = $log.log.logentry | Foreach-Object {
            $logEntry = $_

            $logEntry | Select-Object `
                @{ Name = "Revision"; Expression = { [int]$logEntry.revision } },
                @{ Name = "Author"; Expression = { $logEntry.author } },
                @{ Name = "Date"; 
                   Expression = {
                       if ( $NoFormat )
                       {
                           [datetime]$logEntry.date
                       }
                       else
                       {
                           "{0:dd/MM/yyyy hh:mm:ss}" -f [datetime]$logEntry.date
                       }
                   } },
                @{ Name = "Message"; Expression = { $logEntry.msg } } | 

            Foreach-Object {        
                $_ | where-object { ![string]::IsNullOrEmpty($logEntry.msg) }  | Select-Object  Author, Date, Message              
            }
      }

    return $logObjects
 }

#send email function
 Function SendEmail($SendEmailsTo_, $EmailSubject_, $changes_)
 {
   $emailFrom = "automation@test.com" 
   $smtpserver="smtp.test.com" 
   $smtp=new-object Net.Mail.SmtpClient($smtpServer) 

   foreach ($email in $SendEmailsTo_.split(';'))
   {   
      $smtp.Send($emailFrom, $email, $EmailSubject_, $changes_)
   }
 }

#get svn logs since the last release
$SVNChanges = GetSVNLogs $lastBuildDate

$changes += "`r`n`r`n";
$changes += "SVN changes sice last release`r`n";
$changes += $SVNChanges | Format-Table -AutoSize | Out-String;
$changes += "`r`n-------------------------------------------------------------------`r`n";
$changes += "This is automated email, please do reply directly!";

#send email 
$EmailSubject = "Release $Version changes";

SendEmail $SendEmailsTo $EmailSubject $changes;

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...Loading...

Displaying dynamic javascript charts using MVC

When creating business solutions there is often a need to display dynamic JavaScript charts using specific entity data that can be easily extended throughout the system. Properly designed system should always provide clear separation between business entities and UI layer. When using JavaScript this separation can be easily neglected.
In this article I will show you how to use entity oriented approach when mixing C# and JavaScript code.

In order to do that we will create JavaScript api that will be used to call web controllers such as clientController, productController, userController etc. Each of these controllers will provide us entity related data that can be displayed on the web pages throughout the system. The data will be sent using json protocol along with the target chart configuration we want.

jschart_line

In order to display above charts within our application, we need to simply create div tag that will serve as a placeholder for our chart, we also need to assign attributes and “chartButton” class name to the element that will trigger chart display. The attributes will contain entity name e.g. data-entity=”client”, chart placeholder e.g. targetdiv=”chart1Div” and optionally data parameters e.g. params=”clientId=?&fullInfo=?”

  <input type="button" value="show client chart" class="chartButton" data-entity="client" data-params="clientId=@ViewBag.ClientId&fullInfo=true" data-targetdiv="chart1Div" />
  <input type="button" value="show product chart" class="chartButton" data-entity="product" data-targetdiv="chart1Div" />
  <input type="button" value="show user chart" class="chartButton" data-entity="user" data-targetdiv="chart1Div" />
  <div id="chart1Div"></div>

We also need to add following script references to our _Layout.cshtml file. In our case we will use HighCharts, hence reference to highcharts.js script (we are not limited only to this control, you can convert it to use other JavaScript chart controls as well if needed). The file chartAPI.js will contain our main logic that connects Web Api controllers and JavaScript code.

   <script src="https://code.highcharts.com/highcharts.js" type="text/javascript"></script>
   <script src="@Url.Content("~/Scripts/chartAPI.js")" type="text/javascript"></script>

Let’s start implementation from creating Web Api entity controllers and chart data dto object that will transfer the chart data and it’s configuration.

The ChartData class will contain chart series and chartConfig object – it can be extended at any time depending on our requirements.

   public class ChartData
   {
        public ChartConfig ChartConfig { get; set; }
        public List<Series> Series { get; set; }
    }

    public class Series
    {
        public string Name { get; set; }
        public string Color { get; set; }
        public string DashStyle { get; set; }      
        public List<PointData> Points { get; set; }
    }

    public class PointData
    {
        public DateTime Date { get; set; }
        public double Value { get; set; }
    }

    public class ChartConfig
    {
        public ChartConfig()
        {
            tooltipPointFormat = "";
        }

        public string chartTitle { get; set; }
        public string defaultSeriesType { get; set; }
        public int width { get; set; }
        public int height { get; set; }
        public string tooltipPointFormat { get; set; }
        public bool pie_allowPointSelect { get; set; }
        public bool pie_dataLabelsEnabled { get; set; }
        public bool pie_showInLegend { get; set; }   
    }

Having above classes created, we can now use it to fill it with dummy data inside our entity controller. We can add as many series as we like, we may also configure desired chart’s width, height, series colors etc. Please note that controller’s method will be mapped based on defined html params (see above html configuration).

 public string Get(int clientId, bool fullInfo = true)
 { 
    var data = new ChartData();

    //configure chart
    data.ChartConfig = new ChartConfig()
    {
        chartTitle = "Clients",
        defaultSeriesType = "line",
        width = 800,
        height = 400,
    };

    #region add series
    data.Series = new List<Series>();

    data.Series.Add(new Series()
    {
        Name = "Series1",
        Color = ColorTranslator.ToHtml(Color.Navy),
        DashStyle = "ShortDash",
        Points = new List<PointData>()
    });

    data.Series.Add(new Series()
    {
        Name = "Series2",
        Color = ColorTranslator.ToHtml(Color.Red),
        Points = new List<PointData>()
    });

    data.Series.Add(new Series()
    {
        Name = "Series3",
        Points = new List<PointData>()
    });

    #endregion

    #region add dummy data
    var random = new Random();
    foreach (var serie in data.Series)
    {
        for (var i = 0; i < 50; i++)
        {
            serie.Points.Add(new PointData()
            {
                Date = DateTime.Now.AddDays(-i),
                Value = random.Next(10, 100)
            });
        }
    }
    #endregion

    var json = JsonHelper.ToJSON<ChartData>(data);

    return json;
}

Our chartAPI.js file will contain main JavaScript logic encapsulating data retrieval and constructing the chart object. The function getEntityData simply calls appropriate entity controller using $.getJSON function. When data is retrieved, the renderChart function is used to create chart object, fill it with data and configure according to obtained configuration.

 //this function is used by UI to trigger chart display
$(document).ready(function () {
    //process data on button click
    $(".chartButton").click(function (sender) {

        //read button attributes
        var entity = $(this).attr("data-entity");
        var targetDiv = $(this).attr("data-targetDiv");
        var params = $(this).attr("data-params");

        //call encapsulated function
        getEntityData(entity, params, function (data, status, response) {
            if (status === "success") {
                renderChart(data, targetDiv);
            }
            else if (status === "error") {
                //process the error here if needed

                alert('An error occurred: ' + response);
            }
            else if (status === "complete") {
                //attach your load completed events here if needed

                //alert('Data loaded!');
            }
        });
    });
});

//Gets data for the entity
//This function is encapsulated 
function getEntityData(entityName, params, callbackFn) {
    if (typeof callbackFn != 'function') {
        alert('Incorrect callback function attached!'); return;
    }

    //make sure correct path is set
    var root = location.protocol + "//" + location.host;

    $.getJSON(root + '/API/' + entityName + "?" + params, null)
    .success(function (data, status, response) {
        //create object from json
        var resultObject = JSON.parse(data);
        callbackFn(resultObject, status, response);
    })
    .error(function (e, status, response) {
        callbackFn(e, status, response);
    })
    .complete(function (e, status, response) {
        callbackFn(e, "complete", response);
    });
}

//Renders chart
function renderChart(data, renderTo) {
    var chart1 = new Highcharts.Chart({
        chart: {
            renderTo: renderTo,
            defaultSeriesType: data.ChartConfig.defaultSeriesType,
            width: data.ChartConfig.width,
            height: data.ChartConfig.height,
        },
        title: {
            text: data.ChartConfig.chartTitle,
        },
        tooltip: {
         pointFormat: data.ChartConfig.tooltipPointFormat,
        },
        plotOptions: {
                pie: {
                    allowPointSelect: data.ChartConfig.pie_allowPointSelect,
                    cursor: 'pointer',
                    dataLabels: {
                        enabled: data.ChartConfig.pie_dataLabelsEnabled,
                    },
                    showInLegend: data.ChartConfig.pie_showInLegend,
                },
            },
        xAxis: {
            title: {
                text: 'date'
            },
            type: 'datetime',
        },
        yAxis: {
            title: {
                text: 'value'
            }
        },
    });

    $.each(data.Series, function (index, item) {
        //add series
        chart1.addSeries({
            name: item.name,
            dashStyle: item.DashStyle,
            color: item.Color,
            data: []
        }, false);

        //add data points
        $.each(item.Points, function (index, pointData) {
            chart1.series[chart1.series.length - 1].addPoint([
                Date.parse(new Date(parseInt(pointData.Date.substr(6)))),//format date
                pointData.Value,
            ], false);
        });
    });

    chart1.redraw();
}

It is more than certain that you will have to adjust above solution to your specific requirements. At the same time it should give you a good starting point to implement your own solution based on this example.

I have attached project files below for your convenience.

jsChartMVC

1 Star2 Stars3 Stars4 Stars5 Stars (1 votes, average: 4.00 out of 5)
Loading...Loading...

Entity validation engine using c# expression rules

Validating entity data is the common task when building business applications. Very often happens that we want to separate the rules from the app code to be able to quickly change it without affecting the application. In this article we will build simple validation engine that will be evaluating rules stored as linq expressions.

In our example we will hardcode the rules in separate .cs files (based on entity type). You may also want to store the linq expressions as a string in database and also compile the code dynamically from the string and then execute the rules based on the output dll file.

validation-engine

In our case we will hard code the rule in .css files. The following example shows the rule definitions for Client entity.

  public class ClientValidationRules : BaseValidationRuleSet<Client>
   {
        public ClientValidationRules(Client arg)
        {
            RuleList.Add(new ValidationRule<Client>(
                //set the rule
                a => a.Name.Trim().Any() && !a.Name.Any(b => char.IsLower(b))
                ////
                , arg)
                {
                    //set the result
                    Message = "Name cannot be all in uppercase!",
                    ResultTypeIfFailed = ValidationResultType.ERROR,
                    SuggestionString = arg.Name.ToLower()
                });

            RuleList.Add(new ValidationRule<Client>(
                //set the rule
               a => a.DateOfBirth.AddYears(18) > DateTime.Now
                ////
               , arg)
            {
                //set the result
                Message = "You must be at lest 18 years old to register!",
                ResultTypeIfFailed = ValidationResultType.ERROR,
                SuggestionString = string.Format("Wait {0} days", DateTime.Now.Subtract(arg.DateOfBirth).TotalDays.ToString("0"))
            });
        }
    }
}

You can see that this way allows us to easily define rule’s logic based on the entity data. We can also define suggestion message for each rule separately. By setting ValidationResultType as a rule result we can for example allow to save data if this is only a warning or prevent saving the data by the user if this is an error.

The implementation is as follows. Let’s start with generic ValidationRule class containing the rule data and delegate to run.

 namespace RuleEngine
 {
    public class ValidationRule<T> where T : class
    {
        public string Message { get; set; }
        public string SuggestionString { get; set; }
        
        public ValidationResultType ResultTypeIfFailed { get; set; }

        internal Func<T, bool> RuleDelegate { get; set; }
        internal T ObjectTovalidate { get; set; }

        public ValidationRule(Func<T, bool> rule, T arg)
        {
            RuleDelegate = rule;
            ObjectTovalidate = arg;
        }

        public bool RunRuleDelegate()
        {
            return RuleDelegate(ObjectTovalidate);
        }
    }
 }

It’s base class will have the “Run” method implementation.

 namespace RuleEngine
 {
    public abstract class BaseValidationRuleSet<T> where T : class
    {
        public List<ValidationRule<T>> RuleList { get; internal set; }

        /// <summary>
        /// Initiates rule list
        /// </summary>
        public BaseValidationRuleSet()
        {
            RuleList = new List<ValidationRule<T>>();
        }

        /// <summary>
        /// Run all added rules
        /// </summary>
        /// <returns></returns>
        public ValidationResult Run()
        {
            foreach (var rule in this.RuleList)
            {
                var result = rule.RunRuleDelegate();

                if (result)
                {
                    return new ValidationResult() { IsValid = false, ResultType = rule.ResultTypeIfFailed, Message = rule.Message, SuggestionString = rule.SuggestionString };
                }
            }

            return new ValidationResult() { IsValid = true, ResultType = ValidationResultType.OK };
        }
    }
}

Let’s define ValidationResult class that will be returned after running the rule delegate.

 namespace RuleEngine
 {
    public class ValidationResult
    {
        public bool IsValid { get; set; }
        public string SuggestionString { get; set; }
        public string Message { get; set; }
        public ValidationResultType ResultType { get; set; }
    }
}

The ValidationEngine class will have the entry point for different entity validation methods. You may also want to create generic function to pass the validating object to. You also can preload all rules at runtime and evaluate dynamically using AppDomain.CurrentDomain.GetAssemblies() method.

  public class ValidationEngine: ValidationEngineBase
  {
      public ValidationResult RunProjectRules(Project project)
      {
          return new ProjectValidationRules(project).Run();
      }
      public ValidationResult RunClientRules(Client client)
      {
          return new ClientValidationRules(client).Run();
      }
   }

In order to test our solution we can use console app. We also need to load some fake data into our entities.

     static void Main(string[] args)
        {
            var validationEngine = new ValidationEngine();

            #region load entities
            var client = new Client()
                  {
                      Name = "CLIENT NUMBER1a",
                      DateOfBirth = DateTime.Now.AddYears(-10)
                  };

            var project = new Project()
            {
                Name = "PROJECT NUMBER1",
                StartDate = DateTime.Now.AddDays(1),
                EndDate = DateTime.Now.AddDays(1)
            }; 
            #endregion


            #region validate client
            Console.WriteLine("Starting client validation");

            var result = validationEngine.RunClientRules(client);

            if (!result.IsValid)
            {
                Console.WriteLine(result.Message);
                Console.WriteLine(string.Format("Suggestion: {0}", result.SuggestionString));
            }
            else
            {
                Console.WriteLine("OK");
            }
            #endregion

            Console.WriteLine("--------------------");

            #region validate project
            Console.WriteLine("Starting project validation");

            result = validationEngine.RunProjectRules(project);

            if (!result.IsValid)
            {
                Console.WriteLine(result.Message);
                Console.WriteLine(string.Format("Suggestion: {0}", result.SuggestionString));
            }
            else
            {
                Console.WriteLine("OK");
            }
            #endregion

            Console.ReadLine();
        }

I have included project files bellow for your tests.


#Article update#

We can also create simple business rule engine in very similar way, here is the implementation:

 namespace RuleEngine
 {
    /// <summary>
    /// To be extended if needed
    /// </summary>
    public abstract class BusinessRuleEngineBase
    {
        
    }

    /// <summary>
    /// Runs set of rules for different entities
    /// </summary>
    public class BusinessRuleEngine : BusinessRuleEngineBase
    {
        /// <summary>
        /// Run rules for ScheduledFlight 
        /// </summary>
        /// <param name="scheduledFlight"></param>
        /// <returns></returns>
        public bool RunBusinessRulesFor(ScheduledFlight scheduledFlight)
        {
            return new FlightBusinessRules(scheduledFlight).Run();
        }
    }
 }

Business rule implementation containing evaluating and executing delegate

/// <summary>
/// Runs delegate on the specified business rule set
/// </summary>
/// <typeparam name="T"></typeparam>
public class BusinessRule<T> where T : class
{
    public string RuleName { get; set; }
    internal Func<T, bool> RuleDelegate { get; set; }
    internal Func<T, bool> ExecuteRuleDelegate { get; set; }
    internal T ObjectToValidate { get; set; }
    internal T ObjectToApply { get; set; }

    public BusinessRule(Func<T, bool> rule, Func<T, bool> ruleApply, T arg)
    {
        RuleDelegate = rule;
        ExecuteRuleDelegate = ruleApply;
        ObjectToValidate = arg;
        ObjectToApply = arg;
    }

    /// <summary>
    /// Determines if business rule should be applied
    /// </summary>
    /// <returns></returns>
    public bool ApplyRuleDelegate()
    {
        return RuleDelegate(ObjectToValidate);
    }

    /// <summary>
    /// Applies business rule on the current entity
    /// </summary>
    /// <returns></returns>
    public bool ApplyBusinessDelegate()
    {
        return ExecuteRuleDelegate(ObjectToApply);
    }
    
}

BaseBusinessRuleSet base class

/// <summary>
/// Base class for business rule sets
/// </summary>
/// <typeparam name="T"></typeparam>
public abstract class BaseBusinessRuleSet<T> where T : class
{
    public List<BusinessRule<T>> RuleList { get; internal set; }

    /// <summary>
    /// Initiates business rule list
    /// </summary>
    public BaseBusinessRuleSet()
    {
        RuleList = new List<BusinessRule<T>>();
    }

    /// <summary>
    /// Run all added business rules
    /// </summary>
    /// <returns></returns>
    public bool Run()
    {
        foreach (var rule in this.RuleList)
        {
            var result = rule.ApplyRuleDelegate();

            //if rule is true, run business logic
            if (result)
            {
                rule.ApplyBusinessDelegate();

                return true;
            }
        }

        return false;
    }
}

And finally the business rule set-up class.

public class FlightBusinessRules : BaseBusinessRuleSet<ScheduledFlight>
{
    /// <summary>
    /// Setup business rules for ScheduledFlight
    /// </summary>
    /// <param name="arg"></param>
    public FlightBusinessRules(ScheduledFlight arg)
    {

        RuleList.Add(new BusinessRule<ScheduledFlight>(
            //set the rule
                a => (double)arg.Passengers.Count(p => p.Type == PassengerType.AirlineEmployee) / arg.Passengers.Count > arg.FlightRoute.MinimumTakeOffPercentage
            ,
            //execute rule
                 e =>
                 {
                     //lower the base price till 100
                     if (arg.FlightRoute.BasePrice >= 100)
                     {
                         arg.FlightRoute.BasePrice -= 10;
                     }

                     //add more logic here...

                     return true;
                 },
                arg) { RuleName = "ApplyDiscountFor_AirlineEmployee" });


        RuleList.Add(new BusinessRule<ScheduledFlight>(
            //set the rule
              a => a.FlightRoute.BasePrice == 0
              ,
            //execute rule
               e =>
               {
                   //enforce minimal base price
                   arg.FlightRoute.BasePrice = 100;

                   return true;
               },
              arg) { RuleName = "EnsureMinimalBasePrice" });

    }
}

RuleEngine

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...Loading...