Code Optimization

If you run into this issue trying to profiling your code with Visual Studio Performance Profiler and getting this error:

Unhandled exception. System.TypeLoadException: Could not load type System.Threading.Interlocked

This is because of you enabled the checkbox under the CPU Usage > Collect call counts (.Net Only)

Just disable it and it should start working again.

1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y

Dear friends,

Thank you for comming and here I will show you how to make a huge bulk UPDATE in database from your C# code using Entity Framework.

My issue was about calculation prices for +50K products and then updating these product prices all at once.

Before this optimization SQL Update took more than 60 seconds ( using different approaches and of course you even should not think to do it with EF update/save method at all). After implementation of this approach we get less than 10 seconds for updating +50K products (of course it depends on hosting and environment you use). So what is that?

Table-Valued Parameters

yes, may be you have heard about it but have not understand it enough to use or just have not believed that it would really help you.

So everything is really easy - you create table, put all data you need for your update - it is +50K rows and pass it to your stored procedure that knows how to use it, here is an example:

- create table and columns

                            DataTable tblUpdate = new DataTable();
                            tblUpdate.Columns.Add("Id", typeof(int));               
                            tblUpdate.Columns.Add("DeliveryDateId", typeof(int));   
                            tblUpdate.Columns.Add("OldPrice", typeof(decimal));     
                            tblUpdate.Columns.Add("Price", typeof(decimal));        
                            tblUpdate.Columns.Add("ProductCost", typeof(decimal));  
                            tblUpdate.Columns.Add("AdditionalShippingCharge", typeof(decimal)); 
                            tblUpdate.Columns.Add("DisableBuyButton", typeof(bool));
                            tblUpdate.Columns.Add("Published", typeof(bool));       
                            tblUpdate.Columns.Add("AdminComment", typeof(string));  
                            tblUpdate.Columns.Add("UpdatedOnUtc", typeof(DateTime));

- add rows

...
                            object[] row = new object[10];                            
                            ...
                            ...
                            tblUpdate.Rows.Add(row);
                            ...

- now it is time to push it to SQL

                            var dataSettings = Core.Data.DataSettingsManager.LoadSettings();

                            using (System.Data.SqlClient.SqlConnection conn = new System.Data.SqlClient.SqlConnection(dataSettings.DataConnectionString))
                            {
                                conn.Open();
                                
                                //check if table-valued type is already exists
                                using (System.Data.SqlClient.SqlCommand cmd = new System.Data.SqlClient.SqlCommand(
@"IF TYPE_ID(N'TargetUDT') IS NULL CREATE TYPE [TargetUDT] AS TABLE
(
Id int not null,
DeliveryDateId int not null,
OldPrice decimal(18,4) not null,
Price decimal(18,4) not null,
ProductCost decimal(18,4) not null,
AdditionalShippingCharge decimal(18,4) not null,
DisableBuyButton bit not null,
Published bit not null,
AdminComment nvarchar(max) null,
UpdatedOnUtc datetime not null
);", conn))
                                {
                                    cmd.CommandType = CommandType.Text;
                                    cmd.ExecuteNonQuery();
                                }

                                //create or update stored procedure
                                using (System.Data.SqlClient.SqlCommand cmd = new System.Data.SqlClient.SqlCommand(
@"CREATE OR ALTER PROCEDURE usp_UpdateAllProdPrices
(@tvpUpdateProducts [TargetUDT] READONLY)
AS
BEGIN
UPDATE dbo.Product
SET Product.DeliveryDateId = ec.DeliveryDateId,
Product.OldPrice = ec.OldPrice,
Product.Price = ec.Price,
Product.ProductCost = ec.ProductCost,
Product.AdditionalShippingCharge = ec.AdditionalShippingCharge,
Product.DisableBuyButton = ec.DisableBuyButton,
Product.Published = ec.Published,
Product.AdminComment = ec.AdminComment,
Product.UpdatedOnUtc = ec.UpdatedOnUtc
FROM @tvpUpdateProducts AS ec INNER JOIN dbo.Product
ON dbo.Product.Id = ec.Id
END", conn))
                                {
                                    cmd.CommandType = CommandType.Text;
                                    cmd.ExecuteNonQuery();
                                }

                                //run the stored procedure and pass the all data as table-valued parameter
                                using (System.Data.SqlClient.SqlCommand cmd = new System.Data.SqlClient.SqlCommand(
    @"usp_UpdateAllProdPrices", conn))
                                {
                                    System.Data.SqlClient.SqlParameter tvpParam = cmd.Parameters.AddWithValue("@tvpUpdateProducts", tblUpdate);
                                    tvpParam.SqlDbType = SqlDbType.Structured;
                                    tvpParam.TypeName = "[TargetUDT]";

                                    cmd.CommandType = System.Data.CommandType.StoredProcedure;

                                    cmd.ExecuteNonQuery();

                                }

                                conn.Close();
                            }

That's it guys enjoy your super-fast bulk update :)

Thank you and see you :)

1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y

Hello friends,

You are trying to free up a few GBs on drive C? You tried to clean WinSxS folder, you have tried Disk cleanup util and even the Dism.exe but still need a few GBs of free space? ;)

You got to the right place - your problem is Dropbox.

- Go to the Program Files (x86) > Dropbox folder
- Delete all Client XX.XX folders except the latest one

You will be amaized that it will free up to 2-4 GB of space! :)

Thank you and see you ;)

1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y

Hello friends,

If you are here then you in need of the same as me - split string by comma that is not inside of any brackets or parentheses.

So to do that you need the following piece of code (C# code):

string oval = "Home, Edit, HDMI (4K, ARC), Test, Double 2.0 x1, Multi (F1, (f2-f5, f6, 58), FF, o2), ARG";
string roval = "";
int isOpen = 0;
for (int i = 0; i < oval.Length; i++)
{
    if (isOpen <= 0 && oval[i] == ',')
    {
        roval += ",,";
    }
    else if (isOpen > 0 && (oval[i] == ')' || oval[i] == ']' || oval[i] == '}'))
    {
        isOpen--;
        roval += oval[i];
    }
    else if ((oval[i] == '(' || oval[i] == '[' || oval[i] == '{'))
    {
        isOpen++;
        roval += oval[i];
    }
    else
    {
        roval += oval[i];
    }
}

string[] split = roval.Split(",,", StringSplitOptions.RemoveEmptyEntries);

String you get to split looks like this: 

Home,, Edit,, HDMI (4K, ARC),, Test,, Double 2.0 x1,, Multi (F1, (f2-f5, f6, 58), FF, o2),, ARG

so you need just split it by double comma now.

This is fast, short and clear code that can be easily translated into any other language

Thank you and see you :)

1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y

Are you in need of adding a lot +40K of products, attributes, images, slugs, tags, manufacturers in batch?

Then this post should be same useful for you as for me.

In my case I did require to implement a logic for adding more than 40,000 products with attributes, images, etc. feeding data from a 3rd party API. We will add products to all enabled categories and subcategories that are shown on home page/menu.

So less water more sense :) - here is the step by step explanation:

        [AuthorizeAdmin]
        [Area(AreaNames.Admin)]
        public IActionResult GetProgress()
        {
            //to keep progress
            string progress = string.Empty;
            var storeScope = _storeContext.ActiveStoreScopeConfiguration;
            var directory = "~/wwwroot/images/uploaded/";
            
            string curl = string.Empty;
            //Use direct repo access because of cache problem with re-reading data from settings
            Core.Data.IRepository<Core.Domain.Configuration.Setting> _settingRepository = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Configuration.Setting>>();

            try
            {
                //get list of rest of categories we should add product for
                string ddcat = _settingRepository.Table.SingleOrDefault(s=>s.Name=="DDCategories" && s.StoreId== storeScope)?.Value;
                if (string.IsNullOrEmpty(ddCat))
                {
                    //seems this is the first run
                    var nopallcats = _categoryService.GetAllCategoriesDisplayedOnHomePage(false).Where(c=>c.Published).Select(s => new Category() { ID = s.Id }).ToList();
                    List<int> finalcategories = new List<int>();
                    foreach (var c1 in nopallcats)
                    {
                        var aa1 = _categoryService.GetAllCategoriesByParentCategoryId((int)c1.ID, true);
                        if (aa1.Count > 0)
                        {
                            foreach (var c2 in aa1)
                            {
                                var aa2 = _categoryService.GetAllCategoriesByParentCategoryId(c2.Id, true);

                                if (aa2.Count > 0 && c2.Published)
                                {
                                    foreach (var c3 in aa2)
                                    {
                                        var aa3 = _categoryService.GetAllCategoriesByParentCategoryId(c3.Id, true);
                                        if (aa3.Count <= 0 && c3.Published)
                                        {
                                            finalcategories.Add(c3.Id);
                                        }
                                        else
                                        {
                                            //we have only 3 level of sub categories
                                            throw new ApplicationException("Too many subcategories");
                                        }
                                    }
                                }
                                else
                                {
                                    finalcategories.Add(c2.Id);
                                }
                            }
                        }
                        else
                        {
                            finalcategories.Add((int)c1.ID);
                        }
                    }

                    nopallcats = null;
                    //store remind categories into the settings
                    _settingService.SetSetting<string>("DDCategories", Newtonsoft.Json.JsonConvert.SerializeObject(finalcategories, Newtonsoft.Json.Formatting.None,
                        new Newtonsoft.Json.JsonSerializerSettings()
                        {
                            ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore
                        }), storeScope, true);

                    progress = string.Format("Categories: {0} --", finalcategories.Count);
                    _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));
                }
                else
                {

                    progress = "Start working on category...";
                    _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));
                    List<int> categories = Newtonsoft.Json.JsonConvert.DeserializeObject<List<int>>(ddCat);
                    int products = 0;
                    //take first category to work on
                    var rcategory = categories.FirstOrDefault();
                    //if category id > 0 then lets work on it or exit otherwise
                    if (rcategory != 0)
                    {
                        try
                        {

                            List<Price> prices = null;

                            string responseText = string.Empty;
                            string json = string.Empty;
                            System.Net.HttpWebRequest r = null;
                            System.Net.WebClient webClient = new System.Net.WebClient();
                            lock (Feeder.fileLock)
                            {
                                string pricefile = _fileProvider.GetFileName("dd_prices.json");
                                pricefile = _fileProvider.Combine(_fileProvider.MapPath(dir2), pricefile);
                                //if price file does not exists or older than 60 mins then get new
                                if (!_fileProvider.FileExists(pricefile) || DateTime.UtcNow.Subtract(_fileProvider.GetLastWriteTimeUtc(pricefile)).TotalMinutes > 60)
                                {
                                    responseText = null;
                                }
                                else
                                {

                                    responseText = _fileProvider.ReadAllText(pricefile, Encoding.UTF8);

                                    prices = Newtonsoft.Json.JsonConvert.DeserializeObject<List<Price>>(responseText);
                                    responseText = progress = "re-read prices from file;";
                                    _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));
                                }
                                if (string.IsNullOrEmpty(responseText) && prices == null)
                                {
                                    curl = Feeder.apiUrl + Feeder.getPrices;
                                    r = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(curl);
                                    r.ContentType = "application/x-www-form-urlencoded";
                                    r.Method = "POST";

                                    json = "key=" + Feeder.apiKey;

                                    using (System.IO.StreamWriter sw = new System.IO.StreamWriter(r.GetRequestStream()))
                                    {
                                        sw.Write(json);
                                        sw.Flush();
                                        sw.Close();
                                    }

                                    using (var reader = new System.IO.StreamReader(r.GetResponse().GetResponseStream()))
                                    {
                                        responseText = reader.ReadToEnd();


                                        dynamic dynJson3 = Newtonsoft.Json.JsonConvert.DeserializeObject(responseText);
                                        responseText = null;
                                        prices = new List<Price>();
                                        foreach (var pr in dynJson3.prices)
                                        {
                                            //get prices from JSON and save to own JSON file
                                            dynamic price = ((Newtonsoft.Json.Linq.JToken)pr).First;
                                            prices.Add(new Price()
                                            {
                                                code = price.sku,
                                                id = price.id,
                                                price = price.price,
                                                RPrice = price.RPrice,
                                                delivery = price.delivery,
                                                deliveryText = price.deliveryText
                                            });
                                        }

                                        _fileProvider.WriteAllText(pricefile, Newtonsoft.Json.JsonConvert.SerializeObject(prices, Newtonsoft.Json.Formatting.None,
                                                new Newtonsoft.Json.JsonSerializerSettings()
                                                {
                                                    ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore
                                                }), Encoding.UTF8);


                                        progress = "prices re-downloaded again;";
                                        _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));
                                    }
                                }
                            }

                            
                            responseText = null;

                            var dateranges = _dateRangeService.GetAllDeliveryDates();

                            ProdTemp pt = null;
                            var category = rcategory;
                            int count = 0;
                            int offset = 0, limit = 100, prodProcessed = 0;
                            var ccc = _categoryService.GetCategoryById(category);

                            string temprod = _settingRepository.Table.SingleOrDefault(s => s.Name == "DDProdcts" && s.StoreId == storeScope)?.Value;
                            //create new or re-read last offset object
                            if (string.IsNullOrEmpty(temprod))
                            {
                                pt = new ProdTemp();
                                pt.StoreID = storeScope;
                            }
                            else
                            {
                                pt = Newtonsoft.Json.JsonConvert.DeserializeObject<ProdTemp>(temprod);
                                limit = pt.Limit;
                                offset = pt.Offset;
                                count = pt.TotalProducts;
                                prodProcessed = pt.ProcessedProducts;
                                storeScope = pt.StoreID.GetValueOrDefault();
                            }


                            curl = Feeder.apiUrl + Feeder.getItems;
                            r = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(curl);
                            r.ContentType = "application/x-www-form-urlencoded";
                            r.Method = "POST";

                            //get 3rd party category id to get products for it
                            var ccatid = _genericAttributeService.GetAttribute<int>(ccc, Feeder.gAttrID, storeScope);

                            json = "key=" + Feeder.apiKey + "&limit=" + limit + "&offset=" + offset + "&modifiedDT=" +
                                DateTime.UtcNow.Date.AddMonths(-6).ToString("yyyy-MM-dd", System.Globalization.CultureInfo.InvariantCulture) +
                                "&brand=" + "&category=" + ccatid;
                            using (System.IO.StreamWriter sw = new System.IO.StreamWriter(r.GetRequestStream()))
                            {
                                sw.Write(json);
                                sw.Flush();
                                sw.Close();
                            }
                            responseText = string.Empty;
                            using (var reader = new System.IO.StreamReader(r.GetResponse().GetResponseStream()))
                            {
                                responseText = reader.ReadToEnd();
                            }

                            dynamic dynJson2 = Newtonsoft.Json.JsonConvert.DeserializeObject(responseText);

                            responseText = null;
                            string swm = string.Empty;

                            count = int.Parse((string)dynJson2.count);

                            progress = string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, count, prodProcessed);
                            _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));

                            Core.Domain.Catalog.Product pro = null;
                            dynamic product;
                            Price pprice;

                            var manufRepository = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.Manufacturer>>();
                            var productRepository = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.Product>>();
                            var productTagRepository = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.ProductTag>>();
                            var prodAttrRepo= Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.ProductAttribute>>();
                            var spAttrRepository = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.SpecificationAttribute>>();
                            var spOAttrRepository = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.SpecificationAttributeOption>>();

                            var prodAttrMapRepo = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.ProductAttributeMapping>>();
                            var prodAttrValRepo = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.ProductAttributeValue>>();
                            var prodTagMapRepo = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.ProductProductTagMapping>>();
                            var prodCatMapRepo = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.ProductCategory>>();
                            var prodSAttrMapRepo = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Catalog.ProductSpecificationAttribute>>();

                            string entityName = typeof(Core.Domain.Catalog.Product).Name;
                            var urlRepository = Core.Infrastructure.EngineContext.Current.Resolve<Core.Data.IRepository<Core.Domain.Seo.UrlRecord>>();
                            //stopwatch to get timing
                            System.Diagnostics.Stopwatch swc = new System.Diagnostics.Stopwatch();
                            double totalTimeSec = 0;
                            foreach (var prodd in dynJson2.items)
                            {

                                totalTimeSec = 0;
                                swm = string.Empty;

                                _cache.Set<string>("Progress", string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, count, prodProcessed), TimeSpan.FromMinutes(30));

                                product = ((Newtonsoft.Json.Linq.JToken)prodd).First;

                                int pid = int.Parse((string)Newtonsoft.Json.Linq.JProperty.FromObject(prodd).Name);

                                progress = "pprice | " + string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                                _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));
                                pprice = prices.SingleOrDefault(p => p.id == pid);

                                if (pprice == null)
                                {
                                    pprice = prices.SingleOrDefault(p => p.code == pid.ToString("000000") + "-02");
                                }
                                string pname = product.name;

                                swc.Start();

                                pro = null;
                                //different products may have same SKUs so take care of it
                                pro = _productService.GetProductsBySku(new string[] { (string)product.article }).SingleOrDefault(sk=> sk.Name== pname && sk.ProductCategories.SingleOrDefault(pc=>pc.CategoryId==category)!=null);

                                swc.Stop();
                                totalTimeSec += swc.Elapsed.TotalSeconds;
                                swm = "GetProductBySku: " + swc.Elapsed.TotalSeconds.ToString("0.##");
                                _cache.Set<string>("Progress", progress + swm, TimeSpan.FromMinutes(30));
                                swc.Reset();
                                //we add products if price exists only
                                if (pprice != null && pro == null)
                                {
                                    decimal cost = pprice.price;
                                    decimal price = Feeder.CalculatePrice(cost, pprice.RPrice);

                                    if(price<cost)
                                    {
                                        price = pprice.RPrice;
                                    }
                                    //define max price for category to filter by later
                                    if (pt.MaxPrice < price)
                                        pt.MaxPrice = price;

                                    //manufacturer
                                    swc.Start();
                                    int manuid = product.brand;
                                    progress = "manufs | " + string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                                    _cache.Set<string>("Progress", progress + " " + swm, TimeSpan.FromMinutes(30));
                                    string bname = (string)product.brandName;
                                    var manuf = manufRepository.Table.FirstOrDefault(m => m.Name == bname);
                                    if (manuf == null)
                                    {
                                        manuf = new Core.Domain.Catalog.Manufacturer()
                                        {
                                            Name = product.brandName,
                                            CreatedOnUtc = DateTime.UtcNow,
                                            ManufacturerTemplateId = 1,
                                            MetaDescription = product.brandName,
                                            MetaKeywords = product.brandName,
                                            MetaTitle = product.brandName,
                                            PageSize = 20,
                                            Published = true,
                                            UpdatedOnUtc = DateTime.UtcNow
                                        };
                                        _manufacturerService.InsertManufacturer(manuf);
                                        _storeMappingService.InsertStoreMapping<Core.Domain.Catalog.Manufacturer>(manuf, storeScope);
                                        //store 3rd party manufacturer id to use later
                                        _genericAttributeService.SaveAttribute<int>(manuf, Feeder.gAttrID, manuid);
                                        entityName = typeof(Core.Domain.Catalog.Manufacturer).Name;
                                        //add slug if not exists
                                        if (urlRepository.TableNoTracking.SingleOrDefault(u => u.EntityId == manuf.Id && u.EntityName == entityName && u.LanguageId == 0) == null)
                                        {
                                            var slug1 = _urlRecordService.ValidateSeName(manuf, manuf.Name, manuf.Name, true);
                                            _urlRecordService.InsertUrlRecord(new Core.Domain.Seo.UrlRecord
                                            {
                                                EntityId = manuf.Id,
                                                EntityName = entityName,
                                                LanguageId = 0,
                                                IsActive = true,
                                                Slug = slug1
                                            });
                                        }

                                    }
                                    swc.Stop();
                                    totalTimeSec += swc.Elapsed.TotalSeconds;
                                    swm += ", manufs: " + swc.Elapsed.TotalSeconds.ToString("0.##");
                                    swc.Reset();

                                    //date ranges
                                    swc.Start();
                                    Core.Domain.Shipping.DeliveryDate daterange;
                                    progress = "daterange | " + string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                                    _cache.Set<string>("Progress", progress + " " + swm, TimeSpan.FromMinutes(30));

                                    {
                                        string dt = (string)pprice.deliveryText;
                                        daterange = dateranges.FirstOrDefault(d => d.Name == dt);
                                        if (daterange == null)
                                        {
                                            daterange = new Core.Domain.Shipping.DeliveryDate()
                                            {
                                                Name = pprice.deliveryText
                                            };
                                            _dateRangeService.InsertDeliveryDate(daterange);
                                            dateranges.Add(daterange);
                                        }
                                    }

                                    swc.Stop();
                                    totalTimeSec += swc.Elapsed.TotalSeconds;
                                    swm += ", daterange: " + swc.Elapsed.TotalSeconds.ToString("0.##");
                                    swc.Reset();

                                    //product
                                    swc.Start();
                                    {
                                        //some formatting and length limitation
                                        string shortd = product.descriptionSnort;
                                        if (shortd.Length > 400)
                                        {
                                            shortd = shortd.Substring(0, 399);
                                        }
                                        
                                        if (pname.Length > 400)
                                        {
                                            pname = pname.Substring(0, 399);
                                        }

                                        pro = new Core.Domain.Catalog.Product()
                                        {
                                            ProductType = Core.Domain.Catalog.ProductType.SimpleProduct,
                                            VisibleIndividually = true,
                                            Sku = product.article,
                                            IsShipEnabled = true,
                                            MetaDescription = shortd,
                                            MetaKeywords = shortd,
                                            MetaTitle = pname,
                                            Price = price,
                                            ProductCost = cost,
                                            AdditionalShippingCharge = Feeder.CalculateShipping(price),
                                            CreatedOnUtc = DateTime.UtcNow,
                                            UpdatedOnUtc = ((DateTime)product.modifiedDT).ToUniversalTime(),
                                            Name = pname,
                                            ShortDescription = product.descriptionSnort,
                                            FullDescription = product.description,
                                            BackorderMode = Core.Domain.Catalog.BackorderMode.NoBackorders,
                                            MarkAsNew = true,
                                            AllowBackInStockSubscriptions = true,
                                            AllowCustomerReviews = true,
                                            Published = true,
                                            DeliveryDateId = daterange == null ? 0 : daterange.Id,
                                            LimitedToStores = true, OrderMaximumQuantity=int.MaxValue
                                           ,OrderMinimumQuantity=1, DisableBuyButton = true, ProductTemplateId = 1, ProductManufacturers =
                                        {
                                            new Core.Domain.Catalog.ProductManufacturer() { Manufacturer = manuf }
                                        }
                                            

                                        };
                                        if (!string.IsNullOrEmpty(pro.FullDescription))
                                            pro.FullDescription = pro.FullDescription.Replace("font-size: 1rem;", string.Empty);

                                    }
                                    {
                                        pro.ProductCategories.Add(new Core.Domain.Catalog.ProductCategory()
                                        {
                                            CategoryId = category
                                        });
                                    }

                                    _productService.InsertProduct(pro);

                                    _genericAttributeService.SaveAttribute<int>(pro, Feeder.gAttrID, pid, storeScope);
                                    _genericAttributeService.SaveAttribute<string>(pro, Feeder.gAttrCode, pprice.code, storeScope);

                                    products++;

                                    swc.Stop();
                                    totalTimeSec += swc.Elapsed.TotalSeconds;
                                    swm += ", product: " + swc.Elapsed.TotalSeconds.ToString("0.##");
                                    swc.Reset();

                                    //tags
                                    swc.Start();
                                    progress = "ttag | " + string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                                    _cache.Set<string>("Progress", progress + " " + swm, TimeSpan.FromMinutes(30));
                                    foreach (var tt in pro.Name.Split(' ', StringSplitOptions.RemoveEmptyEntries))
                                    {
                                        string tag = tt.ToUpperInvariant().Trim().Trim(Feeder.trimChars);
                                        if (!string.IsNullOrEmpty(tag))
                                        {
                                            if (manufRepository.Table.Any(m => m.Name == tag))
                                                continue;

                                            var ttag = productTagRepository.Table.FirstOrDefault(t => t.Name == tag);
                                            if (ttag == null)
                                            {
                                                ttag = new Core.Domain.Catalog.ProductTag() { Name = tag };

                                                _productTagService.InsertProductTag(ttag);

                                                pro.ProductProductTagMappings.Add(new Core.Domain.Catalog.ProductProductTagMapping()
                                                {
                                                    ProductTag = ttag,
                                                });


                                                entityName = typeof(Core.Domain.Catalog.ProductTag).Name;

                                                    if (urlRepository.TableNoTracking.SingleOrDefault(u => u.EntityId == ttag.Id && u.EntityName == entityName && u.LanguageId == 0) == null)
                                                    {
                                                        var slug2 = _urlRecordService.ValidateSeName(ttag, ttag.Name, ttag.Name, true);
                                                        _urlRecordService.InsertUrlRecord(new Core.Domain.Seo.UrlRecord
                                                        {
                                                            EntityId = ttag.Id,
                                                            EntityName = entityName,
                                                            LanguageId = 0,
                                                            IsActive = true,
                                                            Slug = slug2
                                                        });
                                                    }
                                                
                                            }
                                            else if (!prodTagMapRepo.Table.Any(a => a.ProductTagId == ttag.Id && a.ProductId == pro.Id))//!pro.ProductProductTagMappings.Any(a => a.ProductTag.Id == ttag.Id))
                                            {
                                                if (!pro.ProductProductTagMappings.Any(a => a.ProductTag.Id == ttag.Id))
                                                {
                                                    pro.ProductProductTagMappings.Add(new Core.Domain.Catalog.ProductProductTagMapping()
                                                    {
                                                        ProductTag = ttag,
                                                    });
                                                }
                                            }
                                        }
                                    }

                                    _productService.UpdateProduct(pro);

                                    swc.Stop();
                                    totalTimeSec += swc.Elapsed.TotalSeconds;
                                    swm += ", ttag: " + swc.Elapsed.TotalSeconds.ToString("0.##");
                                    swc.Reset();
                                    _cache.Set<string>("Progress", progress + " " + swm, TimeSpan.FromMinutes(30));

                                    //pictures
                                    swc.Start();
                                    AddPictures(product, pro);
                                    _productService.UpdateProduct(pro);


                                    swc.Stop();
                                    totalTimeSec += swc.Elapsed.TotalSeconds;
                                    swm += ", pictures: " + swc.Elapsed.TotalSeconds.ToString("0.##");
                                    swc.Reset();
                                    _cache.Set<string>("Progress", progress + " " + swm, TimeSpan.FromMinutes(30));

                                    bool colDef = true;

                                    //attributes
                                    swc.Start();
                                    Core.Domain.Catalog.ProductAttribute pattr = null;
                                    Core.Domain.Catalog.SpecificationAttribute spattr = null;
                                    Core.Domain.Catalog.SpecificationAttributeOption spao = null;
                                    List<Core.Domain.Catalog.ProductSpecificationAttribute> prodSAttrMapRepo_later = new List<Core.Domain.Catalog.ProductSpecificationAttribute>();

                                    dynamic chr;
                                    Newtonsoft.Json.Linq.JProperty cjp;
                                    string val, name, rgb;
                                    System.Drawing.Color c;
                                    progress = "pattr | " + string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                                    _cache.Set<string>("Progress", progress + " " + swm, TimeSpan.FromMinutes(30));
                                    string nlo;
                                    string spaovu;
                                    string spaonu;
                                    foreach (var ch in product.characteristics)
                                    {

                                        chr = ((Newtonsoft.Json.Linq.JToken)ch).First;
                                        cjp = Newtonsoft.Json.Linq.JProperty.FromObject(ch) as Newtonsoft.Json.Linq.JProperty;
                                        val = ((Newtonsoft.Json.Linq.JValue)chr).Value as string;
                                        name = cjp.Name.Trim().Trim(Feeder.trimChars);
                                        if (name.IndexOf('(') >= 0 && name.IndexOf(')') < 0)
                                        {
                                            name += ")";
                                        }
                                        if (name.IndexOf('[') >= 0 && name.IndexOf(']') < 0)
                                        {
                                            name += "]";
                                        }
                                        if (name.IndexOf('{') >= 0 && name.IndexOf('}') < 0)
                                        {
                                            name += "}";
                                        }
                                        nlo = name.ToLowerInvariant();
                                        spaovu = val.ToUpperInvariant();
                                        spaonu = name.ToUpperInvariant();

                                        //work on colors
                                        if (nlo == "цвет" || nlo == "color" || nlo == "колір" || (nlo.Contains("цвет ") || nlo.Contains("color ") || nlo.Contains("колір ")))
                                        {
                                            progress = "pattr | " + string.Format("Categories: {0}, Products: {1}/{2}", categories.Count, (string)dynJson2.count, prodProcessed);
                                            pattr = prodAttrRepo.Table.FirstOrDefault(a => a.Name == name);
                                            if (pattr == null)
                                            {
                                                pattr = new Core.Domain.Catalog.ProductAttribute()
                                                {
                                                    Name = name

                                                };

                                                _productAttributeService.InsertProductAttribute(pattr);

                                            }
                                            string[] colors = spaovu.Split(',', StringSplitOptions.RemoveEmptyEntries);

                                            foreach (var sc in colors)
                                            {
                                                rgb = "#ffffff";
                                                var ssc = sc.Replace(" ", string.Empty).Replace("-", string.Empty).Replace("/", string.Empty).Replace("+", string.Empty).ToLowerInvariant();
                                                var sccc = Feeder.colors.SingleOrDefault(s => s.Item2.Contains(ssc));
                                                if (sccc != null)
                                                    rgb = sccc.Item1;
                                                else
                                                {
                                                    c = System.Drawing.Color.FromName(val);

                                                    rgb = "#" + c.R.ToString("X2") + c.G.ToString("X2") + c.B.ToString("X2");
                                                }

                                                if (!prodAttrMapRepo.Table.Any(a => a.ProductAttributeId == pattr.Id && a.ProductId == pro.Id))
                                                    if (!pro.ProductAttributeMappings.Any(a => a.ProductAttribute.Id == pattr.Id
                                                        && a.ProductAttributeValues.Any(aa => aa.Name.ToUpper() == spaovu)))
                                                    {

                                                        pro.ProductAttributeMappings.Add(new Core.Domain.Catalog.ProductAttributeMapping()
                                                        {
                                                            AttributeControlType = Core.Domain.Catalog.AttributeControlType.ColorSquares,
                                                            IsRequired = colors.Length>1,

                                                            ProductAttribute = pattr,
                                                            ProductAttributeValues =
                                            {
                                                new Core.Domain.Catalog.ProductAttributeValue
                                                {
                                                    AttributeValueType = Core.Domain.Catalog.AttributeValueType.Simple,
                                                    Name = sc, IsPreSelected=colDef, ColorSquaresRgb=rgb
                                                }
                                            }

                                                        });
                                                        colDef = false;
                                                    }
                                            }
                                        }
                                        else //work on specification attributes
                                        {
                                            progress = "spattr | " + string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                                            _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));
                                            spattr = spAttrRepository.Table.FirstOrDefault(a => a.Name == name);
                                            spao = null;
                                            if (spattr == null)
                                            {
                                                spattr = new Core.Domain.Catalog.SpecificationAttribute
                                                {
                                                    Name = name
                                                };
                                                _specificationAttributeService.InsertSpecificationAttribute(spattr);

                                                spao = new Core.Domain.Catalog.SpecificationAttributeOption()
                                                {
                                                    Name = val,
                                                    SpecificationAttribute = spattr
                                                };
                                                _specificationAttributeService.InsertSpecificationAttributeOption(spao);
                                            }
                                            else
                                            {
                                                progress = "spao | " + string.Format("Categories: {0}, Products: {1}/{2}", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                                                _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));

                                                spao = spOAttrRepository.Table.FirstOrDefault(s => s.SpecificationAttributeId == spattr.Id && s.Name == val);
                                                if (spao == null)
                                                {
                                                    progress = "!!!" + progress;
                                                    _cache.Set<string>("Progress", progress + " " + swm, TimeSpan.FromMinutes(30));
                                                    
                                                    {
                                                        spao = new Core.Domain.Catalog.SpecificationAttributeOption()
                                                        {
                                                            SpecificationAttribute = spattr,
                                                            Name = val

                                                        };
                                                        _specificationAttributeService.InsertSpecificationAttributeOption(spao);
                                                    }

                                                   
                                                }
                                            }

                                            if (!prodSAttrMapRepo.Table.Any(a => a.SpecificationAttributeOptionId == spao.Id &&
                                             a.ProductId == pro.Id))
                                                if (!pro.ProductSpecificationAttributes.Any(a => a.SpecificationAttributeOptionId == spao.Id
                                                     && a.SpecificationAttributeOption.SpecificationAttributeId == spattr.Id))
                                                {
                                                    pro.ProductSpecificationAttributes.Add(new Core.Domain.Catalog.ProductSpecificationAttribute()
                                                    {
                                                        //disable filtering by SKU or whatever you need
                                                        AllowFiltering = !excludeParams.Any(a => nlo.ToUpper().Contains(a.ToUpper())),
                                                        ShowOnProductPage = nlo.ToUpper() != excludeParams[0].ToUpper() && nlo.ToUpper() != excludeParams[1].ToUpper(),
                                                        SpecificationAttributeOption = spao,
                                                        AttributeType = Core.Domain.Catalog.SpecificationAttributeType.Option
                                                    });
                                            }
                                        }
                                        _productService.UpdateProduct(pro);
                                    }

                                    entityName = typeof(Core.Domain.Catalog.Product).Name;
                                    var slugp = _urlRecordService.ValidateSeName(pro, pro.Name, pro.Name, true);
                                    urlRepository.Insert(new Core.Domain.Seo.UrlRecord
                                    {
                                        EntityId = pro.Id,
                                        EntityName = entityName,
                                        LanguageId = 0,
                                        IsActive = true,
                                        Slug = slugp
                                    });

                                    swc.Stop();
                                    totalTimeSec += swc.Elapsed.TotalSeconds;
                                    swm += ", pattr: " + swc.Elapsed.TotalSeconds.ToString("0.##");
                                    swc.Reset();
                                    _cache.Set<string>("Progress", totalTimeSec.ToString("0.###") + ":" + progress + " " +  swm, TimeSpan.FromMinutes(30));

                                }
                                else if (pro != null)
                                {
                                    //do whatever you need if product already exists
                                }

                                prodProcessed++;
                                offset++;

                                pt.Limit = limit;
                                pt.Offset = offset;
                                pt.ProcessedProducts = prodProcessed;
                                pt.TotalProducts = count;

                                _settingService.SetSetting<string>("DDProdcts", Newtonsoft.Json.JsonConvert.SerializeObject(pt, Newtonsoft.Json.Formatting.None,
                                new Newtonsoft.Json.JsonSerializerSettings()
                                {
                                    ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore
                                }), storeScope, true);
                            }

                            int icount = dynJson2.items is Newtonsoft.Json.Linq.JArray ? ((Newtonsoft.Json.Linq.JArray)dynJson2.items).Count : ((Newtonsoft.Json.Linq.JObject)dynJson2.items).Count;
                            //check if we have done with that category
                            if (dynJson2 == null || icount < 1 || icount < limit)
                            {
                                if (pt.MaxPrice > 0)
                                {
                                    var cat = _categoryService.GetCategoryById(rcategory);
                                    if (pt.MaxPrice > 50000)
                                    {
                                        cat.PriceRanges = "0-9999;10000-24999;25000-49999;50000-";
                                    }
                                    else if (pt.MaxPrice > 20000)
                                    {
                                        cat.PriceRanges = "0-4999;5000-9999;10000-24999;25000-";
                                    }
                                    else if (pt.MaxPrice > 5000)
                                    {
                                        cat.PriceRanges = "0-4999;5000-9999;10000-";
                                    }
                                    else if (pt.MaxPrice > 1000)
                                    {
                                        cat.PriceRanges = "0-999;1000-3999;4000-";
                                    }
                                    else
                                    {
                                        cat.PriceRanges = "0-249;250-499;500-749;750-";
                                    }
                                    _categoryService.UpdateCategory(cat);
                                }
                                //remove category that we have finished
                                categories.Remove(rcategory);
                                // store offset and everything for the next run (I was re-runing it by an ajax calls)
                                pt.Offset = 0;
                                pt.MaxPrice = 0;
                                pt.TotalProducts = 0;
                                pt.ProcessedProducts = 0;
                                _settingService.SetSetting<string>("DDProdcts", Newtonsoft.Json.JsonConvert.SerializeObject(pt, Newtonsoft.Json.Formatting.None,
                                new Newtonsoft.Json.JsonSerializerSettings()
                                {
                                    ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore
                                }), storeScope, true);
                                progress = string.Format("Categories: {0}, Products: {1}/{2} 'tbc", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                            }
                            else // seems there are more products so we will continue on the same category
                            {
                                _settingService.SetSetting<string>("DDProdcts", Newtonsoft.Json.JsonConvert.SerializeObject(pt, Newtonsoft.Json.Formatting.None,
                                new Newtonsoft.Json.JsonSerializerSettings()
                                {
                                    ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore
                                }), storeScope, true);
                                progress = string.Format("Categories: {0}, Products: {1}/{2} 'tbc", ccc.Name + "/" + categories.Count, (string)dynJson2.count, prodProcessed);
                            }
                        }
                        catch (System.Net.WebException wex)
                        {
                            //LogException
                        }
                        catch (Exception ex)
                        {
                            //LogException
                        }
                    }

                    _cache.Set<string>("Progress", progress, TimeSpan.FromMinutes(30));
                    //seems we have more categories to work on
                    if (categories != null && categories.Count > 0)
                    {
                        _settingService.SetSetting<string>("DDCategories", Newtonsoft.Json.JsonConvert.SerializeObject(categories, Newtonsoft.Json.Formatting.None,
                        new Newtonsoft.Json.JsonSerializerSettings()
                        {
                            ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore
                        }), storeScope, true);
                    }
                    else //seems we have done with all categories
                    {
                        progress = "Done";
                        _settingService.SetSetting<string>("DDProdcts", string.Empty, storeScope, true);
                        _settingService.SetSetting<string>("DDCategories", string.Empty, storeScope, true);
                    }
                }

            }
            catch(Exception ex)
            {
                //LogException
            }

            return Json(new { data = progress });
        }

and the add pictures method

        private void AddPictures(dynamic product, Core.Domain.Catalog.Product pro)
        {
            List<Tuple<string, byte[]>> bdata = new List<Tuple<string, byte[]>>();
            var iurls = ((Newtonsoft.Json.Linq.JArray)product.imageFiles).Select(s => (string)s).ToList();

            System.Threading.Tasks.Parallel.ForEach(iurls, iu =>
            {
                try
                {
                    
                    if (!bdata.Any(a => a.Item1 == iu))
                    {
                        System.Net.WebClient webClient = new System.Net.WebClient();
                        bdata.Add(new Tuple<string, byte[]>(iu, webClient.DownloadData(iu)));
                    }
                }
                catch (Exception ex)
                {
                    //LogException
                }
            }
            );
            
            foreach (var iu in iurls)
            {
                try
                {
                    var data = bdata.SingleOrDefault(s => s.Item1 == iu);

                    if (data != null)
                    {
                        string iufn = _fileProvider.GetFileName(iu);
                        string iufnl = iufn.ToLowerInvariant();
                        {
                            {
                                {
                                    
                                    _pictureService.StoreInDb = false;
                                    Core.Domain.Media.Picture propic = _pictureService.InsertPicture(data.Item2, "image/jpeg", iufn, pro.Name, pro.Name, true);

                                    if (propic != null)
                                    {
                                        pro.ProductPictures.Add(new Core.Domain.Catalog.ProductPicture()
                                        {
                                            Picture = propic
                                        });
                                    }
                                }

                            }
                        }
                    }
                }
                catch (Exception ex)
                {
                    //LogException
                }
            }
            bdata.Clear();
        }

This is not really 100% optimized, especially the DB adding/updating part because of using _services instead of _repositories

but seem I had problems using _repos everywhere and not sure why.

Thank you and see you soon 

1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y

Hello friends,

this time I will show how to add your own message tokens to NopCommerce. For that you need two things: create a message token class and register it with dependencyregistrar.

So here is the code:

    public class FeederOT : IDependencyRegistrar
    {
        public int Order => 1000;

        public void Register(ContainerBuilder builder, ITypeFinder typeFinder, NopConfig config)
        {
            builder.RegisterType<OrderMessageToken>().As<IMessageTokenProvider>().InstancePerLifetimeScope();
        }
    }

    public class OrderMessageToken: Nop.Services.Messages.MessageTokenProvider
    {
        #region Fields

        private readonly CatalogSettings _catalogSettings;
        private readonly CurrencySettings _currencySettings;
        private readonly IActionContextAccessor _actionContextAccessor;
        private readonly IAddressAttributeFormatter _addressAttributeFormatter;
        private readonly ICurrencyService _currencyService;
        private readonly ICustomerAttributeFormatter _customerAttributeFormatter;
        private readonly ICustomerService _customerService;
        private readonly IDateTimeHelper _dateTimeHelper;
        private readonly IDownloadService _downloadService;
        private readonly IEventPublisher _eventPublisher;
        private readonly IGenericAttributeService _genericAttributeService;
        private readonly ILanguageService _languageService;
        private readonly ILocalizationService _localizationService;
        private readonly IOrderService _orderService;
        private readonly IPaymentService _paymentService;
        private readonly IPriceFormatter _priceFormatter;
        private readonly IStoreContext _storeContext;
        private readonly IStoreService _storeService;
        private readonly IUrlHelperFactory _urlHelperFactory;
        private readonly IUrlRecordService _urlRecordService;
        private readonly IVendorAttributeFormatter _vendorAttributeFormatter;
        private readonly IWorkContext _workContext;
        private readonly MessageTemplatesSettings _templatesSettings;
        private readonly PaymentSettings _paymentSettings;
        private readonly StoreInformationSettings _storeInformationSettings;
        private readonly TaxSettings _taxSettings;

        private Dictionary<string, IEnumerable<string>> _allowedTokens;

        #endregion

        #region Ctor

        public OrderMessageToken(CatalogSettings catalogSettings,
            CurrencySettings currencySettings,
            IActionContextAccessor actionContextAccessor,
            IAddressAttributeFormatter addressAttributeFormatter,
            ICurrencyService currencyService,
            ICustomerAttributeFormatter customerAttributeFormatter,
            ICustomerService customerService,
            IDateTimeHelper dateTimeHelper,
            IDownloadService downloadService,
            IEventPublisher eventPublisher,
            IGenericAttributeService genericAttributeService,
            ILanguageService languageService,
            ILocalizationService localizationService,
            IOrderService orderService,
            IPaymentService paymentService,
            IPriceFormatter priceFormatter,
            IStoreContext storeContext,
            IStoreService storeService,
            IUrlHelperFactory urlHelperFactory,
            IUrlRecordService urlRecordService,
            IVendorAttributeFormatter vendorAttributeFormatter,
            IWorkContext workContext,
            MessageTemplatesSettings templatesSettings,
            PaymentSettings paymentSettings,
            StoreInformationSettings storeInformationSettings,
            TaxSettings taxSettings):base(
                catalogSettings,
            currencySettings,
            actionContextAccessor,
            addressAttributeFormatter,
            currencyService,
            customerAttributeFormatter,
            customerService,
            dateTimeHelper,
            downloadService,
            eventPublisher,
            genericAttributeService,
            languageService,
            localizationService,
            orderService,
            paymentService,
            priceFormatter,
            storeContext,
            storeService,
            urlHelperFactory,
            urlRecordService,
            vendorAttributeFormatter,
            workContext,
            templatesSettings,
            paymentSettings,
            storeInformationSettings,
            taxSettings
                )
        {
            this._catalogSettings = catalogSettings;
            this._currencySettings = currencySettings;
            this._actionContextAccessor = actionContextAccessor;
            this._addressAttributeFormatter = addressAttributeFormatter;
            this._currencyService = currencyService;
            this._customerAttributeFormatter = customerAttributeFormatter;
            this._customerService = customerService;
            this._dateTimeHelper = dateTimeHelper;
            this._downloadService = downloadService;
            this._eventPublisher = eventPublisher;
            this._genericAttributeService = genericAttributeService;
            this._languageService = languageService;
            this._localizationService = localizationService;
            this._orderService = orderService;
            this._paymentService = paymentService;
            this._priceFormatter = priceFormatter;
            this._storeContext = storeContext;
            this._storeService = storeService;
            this._urlHelperFactory = urlHelperFactory;
            this._urlRecordService = urlRecordService;
            this._vendorAttributeFormatter = vendorAttributeFormatter;
            this._workContext = workContext;
            this._templatesSettings = templatesSettings;
            this._paymentSettings = paymentSettings;
            this._storeInformationSettings = storeInformationSettings;
            this._taxSettings = taxSettings;
        }

        #endregion

        public override void AddOrderTokens(IList<Token> tokens, Order order, int languageId, int vendorId = 0)
        {
            tokens.Add(new Token("Order.OrderGuid", order.OrderGuid, true));
            base.AddOrderTokens(tokens, order, languageId, vendorId);
        }
    }

So, you create your own class, inherit it from the Nop.Services.Messages.MessageTokenProvider and define your own token inside of the AddOrderTokens method.

After that you register it with the IDependencyRegistrar.

Now you can use your own tokens inside of message templates.

See you there: NopCommerce customization - Price Calculation Service

1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y

Today, I had a task to submit data to an PHP API that takes Json data via HTTP POST using ContentType "application/x-www-form-urlencoded" and format like:

order[phone]=+48733552233&order[name]=First Name Last Name&order[deliveryCost]=50&order[deliveryStockCode]=39931b80-e1c2-11e3-8c4a-0050568002cf&order[comment]=test api&orderItems[0][itemID]=194559-0&orderItems[0][salePrice]=9500&orderItems[0][count]=2&orderItems[1][itemID]=071402-0&orderItems[1][salePrice]=750&orderItems[1][count]=5&key=777777777777777

Hah, crazy right? I have never seen such crazy things before and I had no chance except just take it and create myself.

Google did help me a little bit but in my case there were arrays of objects so I had to modify and extend a code I did find on internet.

So, to transform this:

                    var json = new
                    {
                        key = apiKey,
                        order = new
                        {
                            phone = PhoneNumber,
                            name = LastName + " " + FirstName,
                            comment = Comment,
                            deliveryCost = Math.Round(OrderItems.Sum(s => s.AdditionalShippingCharge), 0, MidpointRounding.ToEven).ToString(),
                            deliveryStockCode = Address
                        },
                        orderItems = OrderItems.Select(s => new
                        {
                            itemID = s.ProdId,
                            salePrice = Math.Round(s.Price, 0, MidpointRounding.ToEven),
                            count = s.Quantity
                        }).ToArray()
                    };

into the form data like I shown before, here is my method (it is rough so I'm sure it is not perfect but I had no time to create it cool and clean and may be will re-write it in the future).

Also, in my case , there were anonymous types so it had to identify them somehow, but as you may know already, anonymous types in C# has no explicit and compilation time type declaration to get its typeof().

        public string JsonToHttpFormString(object request, string separator = ",")
        {
            if (request == null)
                throw new ArgumentNullException("request");

            // Get all properties on the object
            var properties = request.GetType().GetProperties()
                .Where(x => x.CanRead)
                .Where(x => x.GetValue(request, null) != null)
                .ToDictionary(x => x.Name, x => x.GetValue(request, null));

            // Get names for all IEnumerable properties (excl. string)
            var propertyNames = properties
                .Where(x => !(x.Value is string) && ((x.Value is IEnumerable) || (x.Value != null && x.Value.GetType().IsConstructedGenericType && x.Value.GetType().Name.Contains("AnonymousType"))))
                .Select(x => x.Key)
                .ToList();

            // Concat all IEnumerable properties into a comma separated string
            bool isAnonym = false;
            foreach (var key in propertyNames)
            {
                var valueType = properties[key].GetType();
                var valueElemType = valueType.IsGenericType
                                        ? valueType.GetGenericArguments()[0]
                                        : valueType.GetElementType();

                isAnonym = valueType.Name.Contains("AnonymousType");
                if (valueElemType.IsPrimitive || valueElemType == typeof(string) || isAnonym)
                {
                    var enumerable = properties[key] as IEnumerable;
                    if (isAnonym && !valueType.IsArray)
                    {
                        List<string> tempvs = new List<string>();
                        var item = properties[key];
                        // Get all properties on the object
                        var properties2 = item.GetType().GetProperties()
                            .Where(x => x.CanRead)
                            .Where(x => x.GetValue(item, null) != null)
                            .ToDictionary(x => x.Name, x => x.GetValue(item, null));

                        foreach (var kkey in properties2)
                        {
                            var valueType2 = kkey.GetType();
                            var valueElemType2 = valueType2.IsGenericType
                                                    ? valueType2.GetGenericArguments()[0]
                                                    : valueType2.GetElementType();

                            if (valueElemType2.IsPrimitive || valueElemType2 == typeof(string))
                            {
                                tempvs.Add(HttpUtility.UrlEncode(key + "[" + kkey.Key + "]") + "=" + HttpUtility.UrlEncode(kkey.Value.ToString()));
                            }
                        }

                        properties.Remove(key);
                        properties.Add(string.Join("&", tempvs), string.Empty);
                        tempvs.Clear();
                        tempvs = null;
                    }
                    else if (isAnonym && valueType.IsArray)
                    {
                        int i = 0;
                        List<string> tempvs = new List<string>();
                        foreach (var item in enumerable)
                        {
                            // Get all properties on the object
                            var properties2 = item.GetType().GetProperties()
                                .Where(x => x.CanRead)
                                .Where(x => x.GetValue(item, null) != null)
                                .ToDictionary(x => x.Name, x => x.GetValue(item, null));

                            foreach (var kkey in properties2)
                            {
                                var valueType2 = kkey.GetType();
                                var valueElemType2 = valueType2.IsGenericType
                                                        ? valueType2.GetGenericArguments()[0]
                                                        : valueType2.GetElementType();

                                if (valueElemType2.IsPrimitive || valueElemType2 == typeof(string))
                                {
                                    tempvs.Add(HttpUtility.UrlEncode(key + "[" + i + "][" + kkey.Key + "]") + "=" + HttpUtility.UrlEncode(kkey.Value.ToString()));
                                }
                            }
                            i++;
                        }
                        properties.Remove(key);
                        properties.Add(string.Join("&", tempvs), string.Empty);
                        tempvs.Clear();
                        tempvs = null;
                    }
                    else
                    {
                        properties[key] = string.Join(separator, enumerable.Cast<object>());
                    }
                }
            }

            // Concat all key/value pairs into a string separated by ampersand and remove trailing '='
            string res = string.Join("&", properties
                .Select(x => string.Concat(
                    string.IsNullOrEmpty(x.Value.ToString()) ? x.Key : HttpUtility.UrlEncode(x.Key), "=",
                    HttpUtility.UrlEncode(x.Value.ToString())).TrimEnd('='))).TrimEnd('=');

            return res;
        }

Also, it has to do URL encoding to get properly formatted data for form submission style and os we finally could get something like that:

order%5Bphone%5D=%2B+44+%28733%29+55-22-33& order%5Bname%5D=%D0%98%D0%BC%D1%8F+%D0%BF%D0%BE%D0%BB%D1%83% D1%87%D0%B0%D1%82%D0%B5%D0%BB%D1%8F& order%5BdeliveryCost%5D=50&order%5BdeliveryStockCode%5D=39931b80-e1c2-11e3-8c4a-0050568002cf& order%5Bcomment%5D=test+api&orderItems%5B0%5D%5BitemID%5D=194559-0&orderItems%5B0%5D%5BsalePrice%5D=9500&orderItems%5B0%5D%5Bcount%5D=2& orderItems%5B1%5D%5BitemID%5D=071402-0&orderItems%5B1%5D%5BsalePrice%5D=750&orderItems%5B1%5D%5Bcount%5D=5&key=777777777777777

Thank you for reading and see you :)

1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y

Today I'm going to share my experience with c#.net and cross-platform desktop application.

I did write a simple and small windows desktop application under Visual Studio 2017 Community Edition and was wondering if I could run it on Linux Ubuntu and Mac.

The application was just one form and a few combo boxes and buttons app and used Newtonsoft.JSON and web communication. So nothing platform-dependent (except windows forms and controls itself ;) ) and I was almost sure it should work on linux and mac.

First thing I did remember was Mono: https://www.mono-project.com/ and another thing was Gtk#.

I did start with Gtk# and did play a few hours with it. Then I just ported my code to the standard Windows Form Application project and framework .NET Framework 7 just to find out how it would go :)

So I started trying it on Ubuntu. To run simple Windows Forms EXE application on Ubuntu you need mono:

sudo apt-get --assume-yes install mono-runtime

Now I tried to run my exe:

mono MyApp.exe

And I got the following error:

Could not load signature of System.Windows.Forms.Control:GetAccessibilityObjectById due to: Could not load file or assembly or one of its dependencies. assembly:Accessibility, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a type:<unknown type> member:<none>
Unhandled Exception:
System.TypeLoadException: Could not load type 'MyApp.Form1' from assembly 'MyApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.

"Hmm, what does it mean?" As mono should already have all this System.*.dll and Microsoft.*.dll libraries I just deleted everything except EXE and Newtonsoft.Json.dll and tried one more time and I got another exception:

Unhandled Exception:
System.IO.FileNotFoundException: Could not load file or assembly or one of its dependencies.
File name: 'System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

sudo apt-get --assume-yes install mono-complete

sudo apt-get --assume-yes install libappindicator0.1-cil-dev
sudo apt-get --assume-yes install libgtk2.0-cil
sudo apt-get --assume-yes install libmono-posix4.0-cil

some of the libraries were already installed but just in case I write them here anyway.

"Lets run" I thought and tried again. "Wow! It works". Cool!

Sometime it gave me a error even my application ran successfully, something like:

Failed to load module "canberra-gtk-module"

so I did find out that it required one more library:

sudo apt-get install libcanberra-gtk-module libcanberra-gtk3-module

Finally I came to the following - you need to install some libraries and mono itself firstly:

sudo apt-get update
sudo apt-get --assume-yes install mono-runtime
sudo apt-get --assume-yes install mono-complete

sudo apt --assume-yes install libappindicator0.1-cil-dev
sudo apt --assume-yes install libgtk2.0-cil
sudo apt --assume-yes install libmono-posix4.0-cil

After that I was able to run my EXE application from command line (make sure your Terminal is pointed to the EXE folder) like that:

mono MyApp.EXE

And I was really impressed when it brought up windows form with buttons combos etc.

Even more it did communicate good via internet and JSON was working as well :) 

and this is for EXE compiled for .Net Framework 7

Then I did start trying similar things on Mac mini with Mac OS X 10.13 but unfortunately with no luck :(

I did try a few cases and was trying to resolve some issues, missed libraries, etc but unfortunately 

I came up to that that it gave just exception stack trace with errors :(

Not sure if I will continue trying to find out a way to run it under Mac but I any success I will let you know ASAP ;)

Thank you

1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y

There are so many times you need to get a place where your frozen process stuck aren't?

Especially if this is a production server or remote environment. So what to do?

I have faced this problem myself and created a small tool that gets stack trace from process by its name.

We will need the following instruments:

- Microsoft.Diagnostics.Runtime.dll

- System.Management.dll

And some code to find a process and get its stack trace.

        private string CatchStacktrace(string domainName)
        {
            int pid = -1;
            var procs = System.Diagnostics.Process.GetProcessesByName("w3wp");

            var proc = procs.FirstOrDefault(p => GetProcessOwner(p.Id).StartsWith(domainName));
            if (proc == null)
                return string.Format("User Name not found: {0}", domainName);

            pid = proc.Id;

            if (pid < 1)
                return string.Format("Process w3wp.exe not found: {0}", domainName);

            StringBuilder sb = new StringBuilder();
            sb.AppendLine(domainName);
            sb.AppendLine(string.Empty);

            using (var tdata = DataTarget.AttachToProcess(pid, 3000))
            {
                // Dump CLR info
                var clrVersion = tdata.ClrVersions.First();
                var dacInfo = clrVersion.DacInfo;

                sb.AppendLine("# CLR Info");
                sb.AppendLine(string.Format("Version:       {0}", clrVersion.Version));
                sb.AppendLine(string.Format("Filesize:      {0:X}", dacInfo.FileSize));
                sb.AppendLine(string.Format("Timestamp:     {0:X}", dacInfo.TimeStamp));
                sb.AppendLine(string.Format("Dac file:      {0}", dacInfo.FileName));

                sb.AppendLine(string.Empty);

                var runtime = clrVersion.CreateRuntime();
                var appDomain = runtime.AppDomains.First();

                sb.AppendLine(string.Format("# Runtime Info"));
                sb.AppendLine(string.Format("AppDomain:     {0}", appDomain.Name));
                sb.AppendLine(string.Format("Address:       {0}", appDomain.Address));
                sb.AppendLine(string.Format("Configuration: {0}", appDomain.ConfigurationFile));
                sb.AppendLine(string.Format("Directory:     {0}", appDomain.ApplicationBase));

                sb.AppendLine(string.Empty);

                // Dump thread info
                sb.AppendLine("## Threads");
                sb.AppendLine(string.Format("Thread count:  {0}", runtime.Threads.Count));
                sb.AppendLine(string.Empty);
                foreach (var thread in runtime.Threads)
                {
                    sb.AppendLine(string.Format("### Thread     {0}", thread.OSThreadId));
                    sb.AppendLine(string.Format("Thread type:   {0}",
                                            thread.IsBackground ? "Background"
                                          : thread.IsGC ? "GC"
                                          : "Foreground"));
                    var blocks = thread.BlockingObjects;
                    if (blocks != null)
                    {
                        foreach(var block in blocks)
                        {
                            sb.AppendLine(string.Format("Blocked by:  {0}   reason: {1}", block.ToString(), block.Reason.ToString()));
                        }
                    }

                    sb.AppendLine(string.Empty);
                    sb.AppendLine("Stack trace:");
                    foreach (var stackFrame in thread.EnumerateStackTrace())
                    {
                        sb.AppendLine(string.Format("* {0}", stackFrame.DisplayString));
                    }

                    sb.AppendLine(string.Empty);
                    sb.AppendLine(string.Empty);
                }
            }

            return sb.ToString();
        }

        string[] argList = new string[] { string.Empty, string.Empty };
        ManagementObjectCollection processList = null;
        ManagementObjectSearcher searcher = null;
        string query = "Select * From Win32_Process Where ProcessID = ";
        public string GetProcessOwner(int processId)
        {
            searcher = new ManagementObjectSearcher(query + processId);
            processList = searcher.Get();

            foreach (ManagementObject obj in processList)
            {
                argList[0] = string.Empty;
                argList[1] = string.Empty;
                int returnVal = Convert.ToInt32(obj.InvokeMethod("GetOwner", argList));
                if (returnVal == 0)
                {
                    return argList[0];
                }
            }

            return "NO OWNER";
        }

As you know asp.net applications all have w3wp process name and run under its own user name usually with the same name as website name. So to find w3wp process we will pass its username like "ok.unsode.com" to the CatchStacktrace method.

If appropriate w3wp process has been found then you will get something like that:

1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y

Do you need to work with blockchain and bitcoin?

There is a library or even two of them to help you. NBitcoin and QBitNinja will help you.

If you need to track your address for income transactions, for example to check if anyone has paid you for your service or goods.

First thing you need to install those NuGet packages.

Then you need your private key hex bytes and when you get it all then just a few lines of code will provide you with all data you need:

Key key = new Key(new byte[] { /*your private key hex bytes*/ });
var addr = key.PubKey.Hash.GetAddress(Network.Main);
var client = new QBitNinjaClient(Network.Main);
var rbalance = client.GetBalance(addr, true).Result;

After that the rblanace will contain Operations property that will list all operations for your address.

Just go over all items in the rbalance.Operations and find everything you need to confirm or decline your user operation.

Thank you

1vqHSTrq1GEoEF7QsL8dhmJfRMDVxhv2y