On Mon, Mar 19, 2018 at 3:46 PM, Menaka Jayawardena <men...@wso2.com> wrote:
> Hi,
>
> As we discussed in the meeting today[1] (19/03/2018), I modified the
> summary log as follows.
>
> {"Bulk User Import Operation Performed by":"admin","User
> Store":"PRIMARY","Duplicate Users":{"Duplicate User Count":8,"User
> Names":[{"Name":"name1"},{"Name":"name2"},{"Name":"name74"},
> {"Name":"name3"},{"Name":"name3"},{"Name":"name"},{"Name":"
> name83"},{"Name":"name5"}]},"Failed Users":{"Failed User Count":2,"Failed
> Users List":[{"Name":"namsdsa","Cause":"Invalid claim uri has been
> provided: http://wso2.org/claims/ctry"},{"Name":"name2ssss","Cause":"Invalid
> claim uri has been provided: http://wso2.org/claims/emaiaddress"}]}}
>
> And also, we discussed to log the bulk user import summary to the audit
> logs in the following format.
>
> Initiator : admin@carbon.super | Action : Add Role | Target : admin |
> Data : {} | Result
>
Does this audit log gives us the message that a bulk user import happened?
Action 'Add Role' does not imply a bulk user import happened IMO.
Is it possible to introduce an action which clearly conveys the actual
operation that occurred?
>
> The data section will contain the importing user list. As in the
> documentation, we support importing a maximum of 500,000 users at a time.
> So, considering the worse case scenario, if we log these users as well, it
> will eat up the storage very quickly and cause in threat conditions.
>
> So IMO, we do not need to log users that are being imported. Also with the
> Megala's feature [2], as the information is also being logged, I think it's
> enough if we only log the result of the operation with Initiator, Action
> and the Target values.
>
> WDYT?
>
> [1] [IAM] [Discussion] Bulk User Import Improvements
> [2] Discussion on Improving Audit logs Related with User Management
>
> Thanks and Regards,
> Menaka
>
> On Tue, Mar 13, 2018 at 12:45 PM, Dimuthu Leelarathne <dimut...@wso2.com>
> wrote:
>
>>
>>
>> On Tue, Mar 13, 2018 at 11:47 AM, Menaka Jayawardena <men...@wso2.com>
>> wrote:
>>
>>> Hi,
>>>
>>> @Denuwanthi: Yes. It can be done. Please find the summery below.
>>>
>>> SUMMERY :
>>> Bulk User Import Operation Performed by: admin
>>> User Store : PRIMARY
>>> Duplicate user count : 8
>>> .
>>>
>>>
>>> *UI Message Modification.*
>>>
>>> Currently, if an error occurred in the process of performing the Bulk
>>> User Import Operation, the following Error message will be shown.
>>>
>>> *Error occurs while importing usernames. All usernames were not
>>> imported. Last error was : Invalid claim uri has been provided:
>>> http://wso2.org/claims/emaiaddress <http://wso2.org/claims/emaiaddress>*
>>>
>>> But there are multiple errors (Duplicate user etc). In this case, I
>>> think it's better if we show a more generic error with a brief summery and
>>> direct them to view the log file for more information.
>>>
>>> For an example:
>>> Bulk User Import Completed with Errors.
>>> Success user count: x Duplicate user count: y Failed user count: z
>>> Please check the user import log for more detailed information.
>>>
>>
>> +1
>>
>> And in the detail log we can log errors and duplicates.
>>
>> thanks,
>> Dimuthu
>>
>>
>>
>>>
>>> Any ideas, suggestions are highly appreciated.
>>>
>>> Thanks and Regards,
>>> Menaka
>>>
>>> On Tue, Mar 13, 2018 at 9:24 AM, Denuwanthi De Silva <
>>> denuwan...@wso2.com> wrote:
>>>
>>>>
>>>>
>>>> On Mon, Mar 12, 2018 at 4:29 PM, Menaka Jayawardena <men...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Here is an experimental user import summery.
>>>>>
>>>>> SUMMERY :
>>>>> Bulk User Import Operation Performed by: admin
>>>>> Duplicate user count: 8
>>>>> Duplicate Users :
>>>>> name1, name2, name74, name3, name3, name, name83, name5,
>>>>>
>>>>> Failed User Count: 2 Failed Users:
>>>>> Name : namsdsa
>>>>> Cause : Invalid claim uri has been provided:
>>>>> http://wso2.org/claims/ctry
>>>>> Name : name2ssss
>>>>> Cause : Invalid claim uri has been provided:
>>>>> http://wso2.org/claims/emaiaddress
>>>>>
>>>>
>>>> Hi Menaka,
>>>>
>>>> Is it possible to print the user domain in the summary as well? Then
>>>> the information of the userstore the users were imported will be available
>>>> as well.
>>>>
>>>> Thanks,
>>>>
>>>>>
>>>>>
>>>>> The cause string is the standard error which comes from the exception.
>>>>> Do we need to print the stack trace here?
>>>>>
>>>>> Also, there are two BulkUserImport classes (CSVUserBulkImport[1] and
>>>>> ExcelUserBulkImport[2]) and also an unused interface [3] (The classes [1]
>>>>> and [2] are concreet classes).
>>>>>
>>>>> @IAM Team: Is there any particular reason why it kept like this?
>>>>>
>>>>> IMO in this implementation, we could use it to avoid code and method
>>>>> duplication. (By making it an Abstract class)
>>>>>
>>>>> [1] https://github.com/wso2/carbon-identity-framework/blob/maste
>>>>> r/components/user-mgt/org.wso2.carbon.user.mgt/src/main/java
>>>>> /org/wso2/carbon/user/mgt/bulkimport/CSVUserBulkImport.java
>>>>> [2] https://github.com/wso2/carbon-identity-framework/blob/maste
>>>>> r/components/user-mgt/org.wso2.carbon.user.mgt/src/main/java
>>>>> /org/wso2/carbon/user/mgt/bulkimport/ExcelUserBulkImport.java
>>>>> [3] https://github.com/wso2/carbon-identity-framework/blob/maste
>>>>> r/components/user-mgt/org.wso2.carbon.user.mgt/src/main/java
>>>>> /org/wso2/carbon/user/mgt/bulkimport/UserBulkImport.java
>>>>>
>>>>> Thanks and Regards,
>>>>> Menaka
>>>>>
>>>>>
>>>>> On Mon, Mar 12, 2018 at 2:14 PM, Menaka Jayawardena <men...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> [- strategy +Architecture]
>>>>>>
>>>>>>
>>>>>> On Mon, Mar 12, 2018 at 12:21 PM, Menaka Jayawardena <men...@wso2.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Hi Dimuthu,
>>>>>>>
>>>>>>> Are you going to add this log appender by default to the
>>>>>>>> configuration?
>>>>>>>>
>>>>>>> We can add the log appender by default and keep it commented. So
>>>>>>> when the user enables the Bulk User import, he also can enable the log
>>>>>>> appender as well.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Mar 12, 2018 at 12:07 PM, Dimuthu Leelarathne <
>>>>>>> dimut...@wso2.com> wrote:
>>>>>>>
>>>>>>>> Hi Menaka,
>>>>>>>>
>>>>>>>> Are you going to add this log appender by default to the
>>>>>>>> configuration?
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>> Dimuthu
>>>>>>>>
>>>>>>>> On Mon, Mar 12, 2018 at 11:48 AM, Dakshika Jayathilaka <
>>>>>>>> daksh...@wso2.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Ruwan,
>>>>>>>>>
>>>>>>>>> Do we need to log each success? IMO admin will more interest on
>>>>>>>>> failures or duplicates. IMHO we can add detail log on failures and
>>>>>>>>> duplicates and then log the summary which includes the success count.
>>>>>>>>>
>>>>>>>>> WDYT?
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> *Dakshika Jayathilaka*
>>>>>>>>> PMC Member & Committer of Apache Stratos
>>>>>>>>> Associate Technical Lead
>>>>>>>>> WSO2, Inc.
>>>>>>>>> lean.enterprise.middleware
>>>>>>>>> 0771100911 <077%20110%200911>
>>>>>>>>>
>>>>>>>>> On Mon, Mar 12, 2018 at 11:35 AM, Ruwan Abeykoon <ruw...@wso2.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Menaka,
>>>>>>>>>> This is nice feature.
>>>>>>>>>> I would suggest adding one line per each user, before adding, and
>>>>>>>>>> one line for each success, failure(with reason).
>>>>>>>>>> Also add a line who performs this operation. Any trackable
>>>>>>>>>> information of the request for audit purposes.
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Ruwan
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Mar 12, 2018 at 11:21 AM, Menaka Jayawardena <
>>>>>>>>>> men...@wso2.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Currently, when performing bulk user import operation in
>>>>>>>>>>> Identity Server, users face following issues.
>>>>>>>>>>>
>>>>>>>>>>> 1. To check import failed users, need to filter the carbon log
>>>>>>>>>>> file.
>>>>>>>>>>> 2. In UI, it shows only the last error that occurred when
>>>>>>>>>>> importing users.
>>>>>>>>>>>
>>>>>>>>>>> *Requirement Description.*
>>>>>>>>>>> There should be a user friendly way to view the import failed
>>>>>>>>>>> users.
>>>>>>>>>>>
>>>>>>>>>>> As a solution for this, we will provide a new log appender which
>>>>>>>>>>> will log the messages to a separate log file specific for bulk user
>>>>>>>>>>> import.
>>>>>>>>>>> This will help users to easily view the status of the imported
>>>>>>>>>>> users and
>>>>>>>>>>> all the error logs.
>>>>>>>>>>>
>>>>>>>>>>> Also currently, as the operation summery, we only have
>>>>>>>>>>>
>>>>>>>>>>> "Success count: " + successCount + ", Fail count: " + failCount
>>>>>>>>>>> + ", Duplicate count: " + duplicateCount
>>>>>>>>>>>
>>>>>>>>>>> Instead, it would be much effective if we could list the failed
>>>>>>>>>>> and duplicate user names as well.
>>>>>>>>>>>
>>>>>>>>>>> "Success count: " + successCount + ", Fail count: " + failCount
>>>>>>>>>>> + ", Duplicate count: " + duplicateCount
>>>>>>>>>>> "Failed Users : " + [Failed Users List] + "Duplicate Users : " +
>>>>>>>>>>> [Duplicate Users List]
>>>>>>>>>>>
>>>>>>>>>>> WDYT?
>>>>>>>>>>>
>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>> Menaka
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Menaka Jayawardena*
>>>>>>>>>>> *Software Engineer - WSO2 Inc*
>>>>>>>>>>> *Tel : 071 350 5470 <071%20350%205470>*
>>>>>>>>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena
>>>>>>>>>>> <https://lk.linkedin.com/in/menakajayawardena>*
>>>>>>>>>>> *Blog: https://menakamadushanka.wordpress.com/
>>>>>>>>>>> <https://menakamadushanka.wordpress.com/>*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>> *Ruwan Abeykoon*
>>>>>>>>>> *Associate Director/Architect**,*
>>>>>>>>>> *WSO2, Inc. http://wso2.com <https://wso2.com/signature> *
>>>>>>>>>> *lean.enterprise.middleware.*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Dimuthu Leelarathne
>>>>>>>> Director, Rapid Response Team
>>>>>>>>
>>>>>>>> WSO2, Inc. (http://wso2.com)
>>>>>>>> email: dimut...@wso2.com
>>>>>>>> Mobile: +94773661935 <+94%2077%20366%201935>
>>>>>>>> Blog: http://muthulee.blogspot.com
>>>>>>>>
>>>>>>>> Lean . Enterprise . Middleware
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Menaka Jayawardena*
>>>>>>> *Software Engineer - WSO2 Inc*
>>>>>>> *Tel : 071 350 5470 <071%20350%205470>*
>>>>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena
>>>>>>> <https://lk.linkedin.com/in/menakajayawardena>*
>>>>>>> *Blog: https://menakamadushanka.wordpress.com/
>>>>>>> <https://menakamadushanka.wordpress.com/>*
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Menaka Jayawardena*
>>>>>> *Software Engineer - WSO2 Inc*
>>>>>> *Tel : 071 350 5470 <071%20350%205470>*
>>>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena
>>>>>> <https://lk.linkedin.com/in/menakajayawardena>*
>>>>>> *Blog: https://menakamadushanka.wordpress.com/
>>>>>> <https://menakamadushanka.wordpress.com/>*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Menaka Jayawardena*
>>>>> *Software Engineer - WSO2 Inc*
>>>>> *Tel : 071 350 5470 <071%20350%205470>*
>>>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena
>>>>> <https://lk.linkedin.com/in/menakajayawardena>*
>>>>> *Blog: https://menakamadushanka.wordpress.com/
>>>>> <https://menakamadushanka.wordpress.com/>*
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Denuwanthi De Silva
>>>> Senior Software Engineer;
>>>> WSO2 Inc.; http://wso2.com,
>>>> Email: denuwan...@wso2.com
>>>> Blog: https://denuwanthi.wordpress.com/
>>>>
>>>
>>>
>>>
>>> --
>>> *Menaka Jayawardena*
>>> *Software Engineer - WSO2 Inc*
>>> *Tel : 071 350 5470 <071%20350%205470>*
>>> *LinkedIn: https://lk.linkedin.com/in/menakajayawardena
>>> <https://lk.linkedin.com/in/menakajayawardena>*
>>> *Blog: https://menakamadushanka.wordpress.com/
>>> <https://menakamadushanka.wordpress.com/>*
>>>
>>>
>>
>>
>> --
>> Dimuthu Leelarathne
>> Director, Rapid Response Team
>>
>> WSO2, Inc. (http://wso2.com)
>> email: dimut...@wso2.com
>> Mobile: +94773661935 <+94%2077%20366%201935>
>> Blog: http://muthulee.blogspot.com
>>
>> Lean . Enterprise . Middleware
>>
>
>
>
> --
> *Menaka Jayawardena*
> Software Engineer
> WSO2 Inc.
>
> Phone : +94 71 350 5470 <+94%2071%20350%205470>
> LinkedIn : https://lk.linkedin.com/in/menakajayawardena
> Blog : https://menakamadushanka.wordpress.com/
>
>
--
Denuwanthi De Silva
Senior Software Engineer;
WSO2 Inc.; http://wso2.com,
Email: denuwan...@wso2.com
Blog: https://denuwanthi.wordpress.com/
_______________________________________________
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
